adding functionality to di-live

Firstly, its great to hear you are customizing an appliance. If it is a generic open source appliance you might want to update the development wiki, see: http://www.turnkeylinux.org/community/development

Could you explain what you mean by: "upgrade functionality"

We currently don't include deb source packages in the archive, but we do release all sourcecode required to build our packages on http://code.turnkeylinux.org . You can just pull the source and build the deb. I agree its not as easy as apt-get source PACKAGE, but unfortunately will have to do for now.

If there is demand for providing source packages and version control checkout we will look into it, but currently our resources are limited and prefer to dedicate our limited time to appliance development.

With regards to di-live, why would you want to disable the partitioning? I'm interested in your use-case.
To disable it you could just chmod -x di-live.d/42partman, just make sure that the target partition is mounted to /target.

Hope the above helps, let me know if you need anything.

Hi Alon,

Thanks for the quick reply. The appliance is not a generic open source one, however, I'll be happy to share the upgrade functionality when ready.

I believe source packages are important, and could save me (and others) some work: creating the package and keeping the modified package updated with the upstream.

My use-case is quite simple. I'm required to add upgrade functionality to the appliance, and instead of packaging my code, creating local repository, etc, I can do it easily during the installation:
- Check if there is already an installed system
- If so, ask user if he want to upgrade or clean install.
- If upgrade, copy required files somewhere (there are few files only I need to backup)
- If upgrade, skip partitioning (no need to re-partition)
- continue with installation
- if upgrade, restore files.

I have 2 options in mind: create a new di-live package based on the sources, or adding templates to the templates db (manually editing /var/cache/debconf/templates.dat is not a good idea), and manually add the required files to /usr/lib/di-live.d. Problem is - I don't know how to do any of those options.

I'd appreciate some guideness...

Thanks,

-- Uri.

Before we get into technical details of a solution, I am still not sure I fully understand the problem you are trying to solve.
Let me try and reiterate to see if I get it:

- You created a custom appliance image (v1) and installed it on machine X
- You then updated the appliance image (v2) and want to use the installation of the appliance to "upgrade" machine X to use v2

Is this correct?

Exactly.

Tell me again why you don't want to create a local repository and leverage the package management system (apt) for upgrades?

couple of reasons:
- There are no packages of the code, and packing it will take time and effort.
- Appliance should have an easy way for upgrade, using a web interface. I'll have to implement one for it.
- Simpliciy: since there are only few files I need to backup when upgrading, it's much easier adding this functionality to the install process, and I won't need to create an upgrade bundle in addition to the ISO one.

In the future I'll probably create a decent upgrade mechanism using a local repository, but at the moment, I can't afford it.

Not sure how you want to integrate a web interface into di-live, I'm lost on that one.

So basically, what you want the installer to do is something like this:
  - Do a check if a previous version of the image is installed, and if so offer an upgrade option
  - The upgrade mechanism would:
    - Backup files from a predetermined location (to where, the network? another partition? tmpfs?)
    - Continue with regular installation to wipe the old and install the new (just repartition, less error prone, doesn't take long)
    - Restore the backed up files to the newly installed system

If this is what you want, I wouldn't call it upgrade but rather backup.

Am I right in saying that?

When doing a proper upgrade, I'll create a web interface to allow upgrade using apt (not to use di-live).

The addition functionality to the installation is an upgrade. I get an upgraded version, with my data from the old version. It's not a backup. data may need to be modified, of course, but it for the application (my application) to take care of it.

Eventually I hacked the upgrade functionality to di-live by adding 2 files to the squashfs (20backup and 80restore):

20backup:
--------------------------------------------
#!/usr/bin/python

import os
import commands
import sys
import debconf
from common import system

BACKUP_FILES_LIST = []

if __name__ == "__main__":
    partition = commands.getoutput('/bin/list-devices partition').split()[0]
    os.system('mkdir -p /target')
    os.system('mount %s /target' % partition)

    if os.path.exists('/target/etc/version.info'):
        template = 'di-live/upgrade'

        debconf.runFrontEnd()
        db = debconf.Debconf()

        db.capb('backup')
        db.reset(template)
        db.input(debconf.HIGH, template)
        db.go()

        if db.getBoolean(template):
            os.system('echo "Backuping Files: " > /upgrade.log')
            os.system('tar zcvf /backup.tgz -C /target %s >> /upgrade.log 2>&1' % ' '.join(BACKUP_FILES_LIST))
            exit(0)

    os.system('umount /target')
    os.rmdir('/target')
--------------------------------------------

80restore:
--------------------------------------------
#!/usr/bin/python

import os
import sys
from common import target_mounted

def main():
    if target_mounted('/target') and os.path.exists('/backup.tgz'):
     # Restoring...
        os.system('cp /upgrade.log /target/var/log/upgrade.log')
        os.system('echo "Restoring Files: " >> /target/var/log/upgrade.log')
        os.system('tar zxvf /backup.tgz -C /target >> /target/var/log/upgrade.log 2>&1')

if __name__ == "__main__":
    main()
--------------------------------------------

to /var/cache/debconf/templates.dat I added those lines:
--------------------------------------------
Name: di-live/upgrade
Default: true
Description: Would you like to perform an upgrade?
Extended_description: There is an installed version of the appliance. Would you like to perform an upgrade? Choose "No" for a clean install.
Type: boolean
Owners: di-live/upgrade
--------------------------------------------

and to /var/cache/debconf/config.dat:
--------------------------------------------
Name: di-live/upgrade
Template: di-live/upgrade
Value: true
Flags: seen
--------------------------------------------

Not very robust (no error handling), but it is enough at the moment.

-- Uri.

Looks good.

Once you have done some testing, and hopefully added some error handling :) post the updated code and we will look into how we can include this in di-live.

You should also consider populating BACKUP_FILES_LIST from a file on disk, instead of hardcoding, this would make it more generically useful.

Cheers.

I didn't get to the error handling, so nothing new to post. However, I realized that I also need to preseed the di-live with the already-answered-questions in the original installation: partitions, root password, etc.

Is there a way to use di-live with preseed file (like in d-i)?

In theory, preseeding should work the same as in d-i, but to facilitate clean re-runs, the di-live components reset all values.
So even if you do preseed they will be discarded (atleast with the current version).

for example, 42partman-base includes the following:
    preset_debconf(reset=templates)

if you could descibe in detail how you are planning to preseed (via the network, local file, etc.) I will look into changing the default behaviour.

You might be interested to know that di-live is now hosted on github, which will make collaboration a lot easier.
http://www.turnkeylinux.org/forum/general/20091103/selected-code-now-hosted-github