SlapOS Home SlapOS

    How To Move To Md5sum Automatic Update

    How To Move to md5sum automatic update
    • Last Update:2023-08-22
    • Version:004
    • Language:en

    Agenda

    • Learn about update-hash
    • Create buildout.hash.cfg
    • Run update-hash
    • Verification

    Md5sums are necessary to ensure quality of a software release. But during development it can be combersome.

    A tool is available in slapos repository to update automatically md5sums for files you need to track: update-hash

    In order to use this tool a dedicated file must be created containing all path and md5sum of our templates: 'buildout.hash.cfg'. software.cfg will extend this file.

    Here is the commit introducing this functionality to html5as Software release: commit diff

    Prerequisites

    Access Theia

    You can access the Theia using the url and logging in with username and password.

    Create buildout.hash.cfg

    Create buildout.hash.cfg in slapos/software/html5as-base/ directory by right-clicking html5as-base and selecting New File.

    Edit buildout.hash.cfg

    Open buildout.hash.cfg and copy this content into it:

    
    # To learn more about how to generate this file read 
    # ../../README.update-hash.rst
    # THIS IS NOT A BUILDOUT FILE, despite purposedly using a compatible syntax.
    # The only allowed lines here are (regexes):
    # - "^#" comments, copied verbatim
    # - "^[" section beginings, copied verbatim
    # - lines containing an "=" sign which must fit in the following categorie.
    #   - "^\s*filename\s*=\s*path\s*$" where "path" is relative to this file
    #     Copied verbatim.
    #   - "^\s*hashtype\s*=.*" where "hashtype" is one of the values supported
    #     by the re-generation script.
    #     Re-generated.
    # - other lines are copied verbatim
    # Substitution (${...:...}), extension ([buildout] extends = ...) and
    # section inheritance (< = ...) are NOT supported (but you should really
    # not need these here).
    
    [template-cfg]
    filename = instance.cfg.in
    md5sum =
    
    [instance_html5as]
    _update_hash_filename_ = instance_html5as.cfg.in
    md5sum =
    
    [template_nginx_conf]
    _update_hash_filename_ = templates/nginx_conf.in
    md5sum =
    
    [template_launcher]
    _update_hash_filename_ = templates/launcher.in
    md5sum =
    
    [template_mime_types]
    _update_hash_filename_ = templates/mime_types.in
    md5sum =
    
    [template_index_html]
    _update_hash_filename_ = templates/index.html.in
    md5sum =

    The sections names must match the ones in software.cfg. This way by extending this file, the sections key values in software.cfg will default to the value present here.

    No need to put the md5sum, they will be calculated automatically by update-hash

    Extend buildout.hash.cfg in software.cfg

    By extending buildout.hash.cfg, the keys in software.cfg for the sections of the same name will default to the one present in buildout.hash.cfg.

    You have to insert the following under extends : 

    buildout.hash.cfg

    If buildout.hash.cfg has the following sections:

    [template_mime_types]
    _update_hash_filename_ = templates/mime_types.in
    md5sum = 4ef94a7b458d885cd79ba0b930a5727e
    
    [template_index_html]
    _update_hash_filename_ = templates/index.html.in
    md5sum = d57cb01df5941e139b02a2f7bdabcdc8

    And software.cfg is like this:

    [buildout]
    extends =
      buildout.hash.cfg
    
    [download-base]
    recipe = slapos.recipe.build:download
    url = ${:_profile_base_location_}/${:_update_hash_filename_}
    
    [template_mime_types_different_name]
    <= download-base
    
    [template_index_html]
    <= download-base
    

    This is equivalent to having this software.cfg:

    [buildout]
    # nothing extended here but listing details as following:
    [template_index_html]
    recipe = slapos.recipe.build:download
    url = ${:_profile_base_location_}/${:_update_hash_filename_}
    _update_hash_filename_ = templates/index.html.in
    md5sum = d57cb01df5941e139b02a2f7bdabcdc8
    
    [template_mime_types_different_name]
    recipe = slapos.recipe.build:download
    url = ${:_profile_base_location_}/${:_update_hash_filename_}
    
    [template_mime_types]
    _update_hash_filename_ = templates/mime_types.in
    md5sum = 4ef94a7b458d885cd79ba0b930a5727e

    Note : your MD5sum may vary, it's fine. 

    Remove unnecessary keys

    Thanks to extend and macros, download sections are now empty in software.cfg

    [instance_html5as]
    # This section inherit from download-base
    <= download-base
    # Filename and md5sum is defined in buildout.hash.cfg
    
    [template_nginx_conf]
    <= download-base
    
    [template_launcher]
    <= download-base
    
    [template_mime_types]
    <= download-base
    
    [template_index_html]
    <= download-base

    Note : don't forget to delete the md5sum section in [template-cfg]. It should look like the screen above. 

    Run update-hash

    In order to update hash, running update-hash is the only thing that needs to be done.

    $ cd ~/srv/project/slapos/software/html5as-base
    $ ../../update-hash

    Validation

    1. md5sum Mismatch
    2. Updated md5sum
    1. Add a small comment in instance.cfg.in. There will be an error in the log of slapos node software due to md5sum mismatch
    2. Run update-hash. It will successfully compile the software.

    Md5sum Mismatch

    Try to add a comment in instance.cfg.in:

    # Test for update-hash

    Re-compiling the software by

    $ slapos node software --all
    

    The error message is expected because the md5sum in buildout.hash.cfg is not up-to-date:

    2021-10-25 23:20:48 slapos[87599] INFO While:
    2021-10-25 23:20:48 slapos[87599] INFO   Installing template-cfg.
    2021-10-25 23:20:48 slapos[87599] INFO Error: MD5 checksum mismatch for local resource at '/srv/slapgrid/slappart11/srv/project/slapos/software/html5as-base/instance.cfg.in'.
    2021-10-25 23:20:48 slapos[87599] ERROR Failed to run buildout profile in directory '/srv/slapgrid/slappart11/srv/runner/software/1a34da0514503cc3edc1f0ca466a94fb'
    2021-10-25 23:20:48 slapos[87599] INFO Finished software releases.

    Updated Md5sum

    Let's update md5sum by calling update-hash:

    $ cd ~/srv/project/slapos/software/html5as-base
    $ ../../update-hash

    The md5sum of [template-cfg] in buildout.hash.cfg will update.

    Re-compiling the software by

    $ slapos node software --all

    There should be no errors in the output.