Bug 1213

Summary: Move .config into output directory
Product: buildroot Reporter: Will Wagner <will_wagner>
Component: OtherAssignee: Will Wagner <will_wagner>
Status: RESOLVED FIXED    
Severity: enhancement CC: buildroot, nick
Priority: P5    
Version: unspecified   
Target Milestone: ---   
Hardware: PC   
OS: Windows   
Host: Target:
Build:
Attachments: Move .config into output directory
Revised patch following feedback
Patch with correct tabbing
Fixed missed renames
As in "Fixed missed renames" but based on clean BR2010.02
Patch rebased to git master
Patched rebase to master

Description Will Wagner 2010-03-01 19:08:42 UTC
Created attachment 1177 [details]
Move .config into output directory

Currently buildroot places most files generated by the build in the output directory (which can be overriden with O=). However a number of files remain within the buildroot tree.

These are:
.auto.deps (incidentally this no longer seems to be used)
.config
.config.cmd
.config.old
package/config/buildroot-config (which then gets copied into output/build) 

This is a problem if you want to use the same buildroot directory to simultaneously build two different targets to locations elsewhere in the filesystem with O= 

The patch modifies config and the top level Makefile so that all files reside in the output folder. Feedback welcome. Could go further and tidy some things up as well but have not done so. In particular we generate .auto.deps but never seem to use it. Also have not updated package/config/kconfig-to-buildroot2.patch as the previous patch was old and the instructions on how to generate it were incorrect.
Comment 1 Grant Edwards 2010-03-17 15:23:26 UTC
Well done.

The patch works for me.

Doing a build no longer contaminates the source tree which prevented
one from using a single source tree to build for two different
configurations or targets.

And "make distclean" now does what it's supposed to. :)

Comment 2 Peter Korsgaard 2010-03-19 08:44:30 UTC
(In reply to comment #0)

> The patch modifies config and the top level Makefile so that all files reside
> in the output folder. Feedback welcome. Could go further and tidy some things
> up as well but have not done so. In particular we generate .auto.deps but never
> seem to use it. Also have not updated
> package/config/kconfig-to-buildroot2.patch as the previous patch was old and
> the instructions on how to generate it were incorrect.

Thanks, it looks very interesting. I have given it a quick try and see two things I would like to change:

 - The .config location has moved to output/.config if you don't use O=. I would prefer to keep it like it is for non-O= builds (easy to do by adding something like CONFIG_DIR that is either set to $(TOPDIR) or $(O) depending on if O is passed.

 - Some temporary files are still written to TOPDIR, breaking the renames when O= points to another file system:

[pid 28140] rename("/tmp/br/.config", "/tmp/br/.config.old") = 0
[pid 28140] rename(".tmpconfig.28140", "/tmp/br/.config") = -1 EXDEV (Invalid cross-device link)
[pid 28140] write(2, "\n*** Error during writing of the"..., 59
*** Error during writing of the Buildroot configuration.

I'll have a closer look at fixing this next week, unless someone beats me to it.
Comment 3 Will Wagner 2010-04-01 16:04:29 UTC
Created attachment 1453 [details]
Revised patch following feedback

Have updated the patch to fix both the issues.

Change to have .config files in buildroot dir id O= not specified was simple.

Modifying config to put temp files in O= folder was relatively straight forward, although obviously it is straying quite a long way from the original kernel source.
Comment 4 Will Wagner 2010-04-01 16:06:42 UTC
Comment on attachment 1453 [details]
Revised patch following feedback

Sorry this patch is bogus, my text editor has converted all the tabs to spaces.

Will fix and attach a new patch
Comment 5 Will Wagner 2010-04-01 16:10:51 UTC
Created attachment 1459 [details]
Patch with correct tabbing

This patch has suggested changes as well as the correct tabbing :)
Comment 6 Nick Leverton 2010-04-09 10:42:29 UTC
Good work, thanks !  Works for make menuconfig; make; make clean.  There still seems to be a cross-device issue with "make oldconfig" though.  I'd like to be able to check out a project's .config file into an empty directory, and just do something like "make O=directory_name oldconfig && make O=directory_name ".

3737  11:32:19.501549 open("/var/tmp/buildroot.test/.config", O_RDONLY) = 3
3737  11:32:19.501760 fstat64(3, {st_mode=S_IFREG|0644, st_size=18954, ...}) = 0
3737  11:32:19.501906 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40020000
3737  11:32:19.502030 read(3, "#\n# Automatically generated make "..., 4096) = 4096
3737  11:32:19.502265 read(3, "ERSION_4_3_3 is not set\n# BR2_GCC"..., 4096) = 4096
3737  11:32:19.502470 read(3, "\n# BR2_PACKAGE_ARGUS is not set\n#"..., 4096) = 4096
3737  11:32:19.502649 read(3, "NT is not set\n# BR2_PACKAGE_USBUT"..., 4096) = 4096
3737  11:32:19.502878 read(3, "not set\n# BR2_TARGET_ROOTFS_INITR"..., 4096) = 2570
3737  11:32:19.503029 read(3, ""..., 4096) = 0
3737  11:32:19.505067 close(3)          = 0
3737  11:32:19.505182 munmap(0x40020000, 4096) = 0
3737  11:32:19.506235 getpid()          = 3737
3737  11:32:19.506343 open(".tmpconfig.3737", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
3737  11:32:19.506523 time(NULL)        = 1270809139

...snips....

3737  11:32:19.510083 write(3, "#\n# Automatically generated make "..., 4096) = 4096
3737  11:32:19.510371 write(3, "ERSION_4_3_3 is not set\n# BR2_GCC"..., 4096) = 4096
3737  11:32:19.510633 write(3, "\n# BR2_PACKAGE_ARGUS is not set\n#"..., 4096) = 4096
3737  11:32:19.511038 write(3, "NT is not set\n# BR2_PACKAGE_USBUT"..., 4096) = 4096
3737  11:32:19.511275 write(3, "not set\n# BR2_TARGET_ROOTFS_INITR"..., 2570) = 2570
3737  11:32:19.511410 close(3)          = 0
3737  11:32:19.511538 munmap(0x40027000, 4096) = 0
3737  11:32:19.511657 rename("/var/tmp/buildroot.test/.config", "/var/tmp/buildroot.test/.config.old") = 0
3737  11:32:19.511900 rename(".tmpconfig.3737", "/var/tmp/buildroot.test/.config") = -1 EXDEV (Invalid cross-device link)
3737  11:32:19.512050 write(2, "\n*** Error during writing of the "..., 59) = 59
3737  11:32:19.512202 exit_group(1)     = ?
3711  11:32:19.512471 <... wait4 resumed> [{WIFEXITED(s) && WEXITSTATUS(s) == 1}], 0, NULL) = 3737
Comment 7 Will Wagner 2010-04-09 16:46:28 UTC
Created attachment 1495 [details]
Fixed missed renames

Your right, I missed a case. I also noticed I managed to build .auto.deps wrong (although I still think no one ever uses it).

Please try this patch out, hopefully all done now.
Comment 8 Nick Leverton 2010-04-13 16:13:48 UTC
That's good, works fine !  Although there was an error in applying it against BR2010.02 - it looks as if it might have been generated against an already part-patched source.  I'm attaching a version with the extra lines removed, which applied with just a few offsets.  Would you have a look at it for me please ?
Comment 9 Nick Leverton 2010-04-14 10:56:47 UTC
Created attachment 1507 [details]
As in "Fixed missed renames" but based on clean BR2010.02
Comment 10 Will Wagner 2010-04-14 11:23:01 UTC
Created attachment 1513 [details]
Patch rebased to git master

Nick,

The patch is meant to patch against latest git master, not 2010.2.

Have updated to latest git, and added Ack from you.

Peter, could this get another review from you? I think everything is now fixed - should I mark the bug fixed? Presumably you'll do that when you commit it?
Comment 11 Will Wagner 2010-06-09 18:25:34 UTC
Created attachment 2005 [details]
Patched rebase to master

Updated patch rebase to latest master. Not tested the gconf option but I think it should just work.

Peter, any chance of inclusion this development cycle?
Comment 12 Peter Korsgaard 2010-06-09 19:12:26 UTC
(In reply to comment #11)

> Updated patch rebase to latest master. Not tested the gconf option but I think
> it should just work.

Thanks.
 
> Peter, any chance of inclusion this development cycle?

Yes, certainly. I'll test it a bit and commit if/when I'm happy with it.
Comment 13 Peter Korsgaard 2010-06-21 21:08:48 UTC
Committed, thanks. There were a few small issues regarding make <blah>_defconfig, but I fixed that.