Amidst rising concerns of software supply chain attacks, the Software Bill of Materials (SBOM) has emerged as a pivotal tool, offering a detailed listing of software components to manage vulnerabilities, dependencies, and licensing. While significant momentum has been gained in standardizing and promoting SBOMs, the critical aspect of their generation, which directly influences the efficacy of their implementation, remains inadequately examined. We conducted a rigorous evaluation of prevailing SBOM generators in the market.

• Our evaluation reveals significant deficiencies in current SBOM generators. We also conduct a comprehensive case study to uncover how each SBOM generator detects dependencies and vulnerabilities during the generation process.
• We develop best practices for developing SBOM generators and a benchmark to facilitate their development

By introducing a benchmark tailored to assess attributes like accuracy, comprehensiveness, and integration capability, our findings aim to guide users in their selection and inspire enhancements in SBOM generator development.