Bug 15961 - empty file statistic on partial build
Summary: empty file statistic on partial build
Status: RESOLVED MOVED
Alias: None
Product: buildroot
Classification: Unclassified
Component: Other (show other bugs)
Version: 2023.11
Hardware: PC Linux
: P5 minor
Target Milestone: ---
Assignee: unassigned
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-02-28 15:46 UTC by Laurent Badel
Modified: 2024-06-15 15:24 UTC (History)
2 users (show)

See Also:
Host:
Target:
Build:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Laurent Badel 2024-02-28 15:46:20 UTC
There seems to be a problem with the package statistics in pkg-generic.mk. 
As I understand the files-lists* files are used to track which files were installed in which directories during the installations steps. 
I have found this quite helpful to keep track version numbers of packages in my final rootfs, for example, but it seems that this functionality is now partly broken and produces only empty files in case of a partial build. 

Looking at pkg-generic.mk, the logic seems to be that we collect lists of files after the configuration step, and store these lists in files-list*.before. Then, at the end of the installation step, we generate new lists of files in files-list*.after, which we compare with the .before files generated earlier, and extract the entries that are present only in the .after file.

The problem is that this does not work when using -rebuild or -reinstall targets, because (i) the .before files are deleted after the first installation, and (ii) the contents of the .before file might not be up-to-date anyway since other stuff may have been installed in-between.

It seems to me that moving the calls to pkg_size_before into the .stamp_*_installed targets themselves would solve the problem. In this way, the state of the directories just before, and just after, the package installation, will be compared, producing the expected results.

I have tested this with the below patch, and the results were as I expected. I would gladly submit it directly, but I am unsure if my understanding of the purpose and contents of these files is correct, and I'd rather not spam the entire list with nonsense.

I am using the 2023.11 tag but I don't see any recent patch indicating that this might have been fixed. 

Thank you very much in advance,

Laurent 

From 99421a92a8dce34385c4ec3d51ee4a1c54327a54 Mon Sep 17 00:00:00 2001
From: Laurent Badel <laurentbadel@eaton.com>
Date: Wed, 28 Feb 2024 15:39:29 +0000
Subject: [PATCH] package/pkg-generic: fix file statistics for partial builds

Signed-off-by: Laurent Badel <laurentbadel@eaton.com>
---
 package/pkg-generic.mk | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 577a148c..db5cc81a 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -265,10 +265,6 @@ $(BUILD_DIR)/%/.stamp_configured:
 	$(Q)mkdir -p $(HOST_DIR) $(TARGET_DIR) $(STAGING_DIR) $(BINARIES_DIR)
 	$(call prepare-per-package-directory,$($(PKG)_FINAL_DEPENDENCIES))
 	$(foreach hook,$($(PKG)_POST_PREPARE_HOOKS),$(call $(hook))$(sep))
-	@$(call pkg_size_before,$(TARGET_DIR))
-	@$(call pkg_size_before,$(STAGING_DIR),-staging)
-	@$(call pkg_size_before,$(BINARIES_DIR),-images)
-	@$(call pkg_size_before,$(HOST_DIR),-host)
 	$(foreach hook,$($(PKG)_PRE_CONFIGURE_HOOKS),$(call $(hook))$(sep))
 	$($(PKG)_CONFIGURE_CMDS)
 	$(foreach hook,$($(PKG)_POST_CONFIGURE_HOOKS),$(call $(hook))$(sep))
@@ -289,6 +285,7 @@ $(BUILD_DIR)/%/.stamp_built::
 $(BUILD_DIR)/%/.stamp_host_installed:
 	@$(call step_start,install-host)
 	@$(call MESSAGE,"Installing to host directory")
+	@$(call pkg_size_before,$(TARGET_DIR))
 	$(foreach hook,$($(PKG)_PRE_INSTALL_HOOKS),$(call $(hook))$(sep))
 	+$($(PKG)_INSTALL_CMDS)
 	$(foreach hook,$($(PKG)_POST_INSTALL_HOOKS),$(call $(hook))$(sep))
@@ -318,6 +315,7 @@ $(BUILD_DIR)/%/.stamp_host_installed:
 $(BUILD_DIR)/%/.stamp_staging_installed:
 	@$(call step_start,install-staging)
 	@$(call MESSAGE,"Installing to staging directory")
+	@$(call pkg_size_before,$(TARGET_DIR))
 	$(foreach hook,$($(PKG)_PRE_INSTALL_STAGING_HOOKS),$(call $(hook))$(sep))
 	+$($(PKG)_INSTALL_STAGING_CMDS)
 	$(foreach hook,$($(PKG)_POST_INSTALL_STAGING_HOOKS),$(call $(hook))$(sep))
@@ -360,6 +358,7 @@ $(BUILD_DIR)/%/.stamp_staging_installed:
 $(BUILD_DIR)/%/.stamp_images_installed:
 	@$(call step_start,install-image)
 	@$(call MESSAGE,"Installing to images directory")
+	@$(call pkg_size_before,$(BINARIES_DIR),-images)
 	$(foreach hook,$($(PKG)_PRE_INSTALL_IMAGES_HOOKS),$(call $(hook))$(sep))
 	+$($(PKG)_INSTALL_IMAGES_CMDS)
 	$(foreach hook,$($(PKG)_POST_INSTALL_IMAGES_HOOKS),$(call $(hook))$(sep))
@@ -370,6 +369,7 @@ $(BUILD_DIR)/%/.stamp_images_installed:
 $(BUILD_DIR)/%/.stamp_target_installed:
 	@$(call step_start,install-target)
 	@$(call MESSAGE,"Installing to target")
+	@$(call pkg_size_before,$(TARGET_DIR))
 	$(foreach hook,$($(PKG)_PRE_INSTALL_TARGET_HOOKS),$(call $(hook))$(sep))
 	+$($(PKG)_INSTALL_TARGET_CMDS)
 	$(if $(BR2_INIT_SYSTEMD),\
-- 
2.17.1
Comment 1 Yann E. MORIN 2024-06-15 15:24:36 UTC
Thank you for your report.

The issue tracker for the Buildroot project has been moved to
the Gitlab.com issue tracker:
    https://gitlab.com/buildroot.org/buildroot/-/issues

We are taking this opportunity to close old issues in this old
tracker. If you believe your issue is still relevant, please
open one in the new issue tracker.

Thank you!