how-to use scmproj for multiple Drupal CMS client sites

Asked by adrianrf@gmail.com

hi, am total newb -- to a) VCS and b) bzr...

am hoping you can recommend a bzr/scmproj configuration scenario for my use-case.

like many semi-technical people, I develop & maintain several websites (for different clients, hosted on different servers) based on Drupal CMS (drupal.org). Drupal is a rapidly-growing PHP/MySQL content management platform with significant support from IBM, Sun, Google, etc.

I am confident what I'm trying to do is a fairly common scenario, applying to thousands of independent Drupal developers.
not many Drupal admins/developers are using bzr yet; if you can help me get this going, I'll write it up and publish it at drupal.org.

Drupal distributions have a core codebase, plus a "multisite" domain-specific directory tree that allows one to add 3rd-party contributed modules to extend (or override) the core Drupal distro's functionality -- **if** the file structure is hosted on a single webserver.

the Drupal directory tree looks like this:

              <drupal root/ (i.e. public_html/)>
                .htaccess [frequently has server-specific tweaks]
                index.php [core code]
                      .. etc.
                modules/ [standard distro modules]
                  moduleA/
                  moduleB/
                      .. etc.
                scripts/ [another core directory]
                      .. etc.
                sites/ [top of Drupal's "multisite" tree]
                  all/ [added items common to all domains]
                    modules/ [third-party functionality contributions]
                      moduleX/
                      moduleY/
                    themes/ ["skins" common to all sites]
                      admin_menu/
                  domain1.com/ [domain1-specific items (i.e. commissioned / third-party contributions]
                    modules/
                      moduleF/
                      moduleG/
                    themes/
                      domain1_theme/ [custom theme for domain1]

                  domain2.org/ [domain2-specific items (i.e. commissioned / third-party contributions]
                    modules/
                      moduleG/
                      moduleM/
                    themes/
                      domain2_theme/ [custom theme for domain2]

                  domain3.net/ [domain3-specific items (i.e. commissioned / third-party contributions]
                    modules/
                      moduleN/
                      moduleO/
                    themes/
                      domain3_theme/ [custom theme for domain3]

                  .. etc.
                themes/ [core distro's themes]
                xmlrpc.php [yet more core code]

if all my sites were hosted on a single server, I think I'd understand how to set up bzr - because I'd only need one repository, and one URL to push to. but since I need each domain to get a) the core distribution files, b) the sites/all/ tree, and c) its respective domainX.TLD/ tree, and push to that domain's unique server URL, I'm seriously confused about the best approach.

considerations:
  Drupal.org releases minor updates to its core code distro quite frequently (say monthly), e.g. fixing security issues.
  3rd-party contributed modules are updated on independent schedules by their various developers. Changes can happen anywhere from annually through daily.
  right now, I'm the only programmer; but I want to involve other developers to help me with domain-specific projects [generally writing domain-specific modules / themes]

what I'm looking to bzr/scmproj to help me with:
  merge in site-specific development work done by me or by other developers
  testing new versions of Drupal core code in a local "sandbox"
  testing new versions of third-party contributions [whether they're shared across sites, or are site-specific] in a local sandbox
  share the testing work for a given domain with other developers
  push a new set of core code + third-party contributions to the production server for a specific domain, after sandbox testing is done
  quickly roll back changes to a given domain (or all domains, if there's a problem with some code they all share), if/when problematic issues emerge

if you can help steer my thinking, I'd would be very grateful!

best,

Adrian Russell-Falla

Question information

Language:
English Edit question
Status:
Answered
For:
ScmProj plugin Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Alexander Belchenko (bialix) said :
#1

<email address hidden> пишет:
> New question #65246 on ScmProj plugin:
> https://answers.launchpad.net/bzr-scmproj/+question/65246
>
> hi, am total newb -- to a) VCS and b) bzr...

Well, some basic knowledge of bzr required here, at least you need to know how
to work with separate branches.

Basic thing about bzr you need to know: bzr working with branches. Every branch
has repository (as internal storage for history) and could have working tree with
your actual working files.

Basic thing about scmproj you need to know: scmproj working with collection of branches
as one big project. In the same time every branch in such project is still plain
bzr branch and all typical operations applicable to every separate component.
Scmproj is only provide the way to automatize some operations on the set of branches,
i.e. run some command for all components at once.

> am hoping you can recommend a bzr/scmproj configuration scenario for my use-case.

If your intent is to work on set of branches then scmproj certainly will help you.

> Drupal distributions have a core codebase,
 > plus a "multisite" domain-specific directory tree that allows one to add 3rd-party contributed
 > modules to extend (or override) the core Drupal distro's functionality --
 > **if** the file structure is hosted on a single webserver.

OK, every bzr branch should have their own directory, nested branches is supported well
by bzr and scmproj. You need to decide what modules/distros should go in which branch.

> the Drupal directory tree looks like this:
>
> <drupal root/ (i.e. public_html/)>
> .htaccess [frequently has server-specific tweaks]
> index.php [core code]
> .. etc.
> modules/ [standard distro modules]
> moduleA/
> moduleB/
> .. etc.
> scripts/ [another core directory]
> .. etc.
> sites/ [top of Drupal's "multisite" tree]
> all/ [added items common to all domains]
> modules/ [third-party functionality contributions]
> moduleX/
> moduleY/
> themes/ ["skins" common to all sites]
> admin_menu/
> domain1.com/ [domain1-specific items (i.e. commissioned / third-party contributions]
> modules/
> moduleF/
> moduleG/
> themes/
> domain1_theme/ [custom theme for domain1]
>
> domain2.org/ [domain2-specific items (i.e. commissioned / third-party contributions]
> modules/
> moduleG/
> moduleM/
> themes/
> domain2_theme/ [custom theme for domain2]
>
> domain3.net/ [domain3-specific items (i.e. commissioned / third-party contributions]
> modules/
> moduleN/
> moduleO/
> themes/
> domain3_theme/ [custom theme for domain3]
>
> .. etc.
> themes/ [core distro's themes]
> xmlrpc.php [yet more core code]
>
> if all my sites were hosted on a single server, I think I'd understand how to set up bzr - because I'd only need one repository, and one URL to push to. but since I need each domain to get a) the core distribution files, b) the sites/all/ tree, and c) its respective domainX.TLD/ tree, and push to that domain's unique server URL, I'm seriously confused about the best approach.
>
> considerations:
> Drupal.org releases minor updates to its core code distro quite frequently (say monthly), e.g. fixing security issues.
> 3rd-party contributed modules are updated on independent schedules by their various developers. Changes can happen anywhere from annually through daily.
> right now, I'm the only programmer; but I want to involve other developers to help me with domain-specific projects [generally writing domain-specific modules / themes]

OK. So first at all you need to divide the big tree above into smaller components.
I'd suggest make drupal core files (top-level directory) as one branch,
and all separate components in sites/ should be separate branches.
I'm not sure does sites/all/ should be part of Drupal core or separate branch
(I'm not familiar with Drupal internals).

> what I'm looking to bzr/scmproj to help me with:

So again, scmproj will help to manage set of branches, can be useful to recreate your
directory structure for development or deployment.

> merge in site-specific development work done by me or by other developers

Can be done as usual bzr merge for 2 branches, scmproj is not related here.

> testing new versions of Drupal core code in a local "sandbox"

Can be done with scmproj (commands project-get, project-update)

> testing new versions of third-party contributions [whether they're shared across sites, or are site-specific] in a local sandbox

Can be done with or without scmproj, because you want to inspect/testing one separate branch.

> share the testing work for a given domain with other developers

Can be done with scmproj (commands project-get to recreate project structure)

> push a new set of core code + third-party contributions to the production server for a specific domain, after sandbox testing is done

Can be done with scmproj (commands project-publish or project-cmd)

> quickly roll back changes to a given domain (or all domains, if there's a problem with some code they all share), if/when problematic issues emerge

Can be done with scmproj (command project-cmd)

> if you can help steer my thinking, I'd would be very grateful!

It seems doable and possible to do with scmproj.
But your question is very big and have several subquestion inside.
Format of Launchpad.net answers service is not very good for such things.

I suggest to continue this discussion in main bzr mailing list, if you don't mind.
I've cc'd this answer to bzr ML as well. You can use gmane to read and post.
(http://dir.gmane.org/gmane.comp.version-control.bazaar-ng.general)

>
> best,
>
> Adrian Russell-Falla

Revision history for this message
adrianrf@gmail.com (adrianrf) said :
#2

Alexander:

many many thanks!
I greatly appreciate your patience with my long question.

so it seems like scmproj is the best answer to deal with administering *all*
the different sites (and their respective branches) as a group -- eg. on
getting a new Drupal distro release, to create sandboxes for each one; and
after testing each, to to one big push up to all the different production
servers associated with the different sites. that's very cool indeed!

am going to check out the main scmproj discussion list now.

best,

Adrian
Adrian Russell-Falla
503-381-4678 [mob]
gtalk/skype/jajah:adrianrf aim:zxadrian
http://www.linkedin.com/in/adrianrf

On Wed, Mar 25, 2009 at 01:42, Alexander Belchenko <
<email address hidden>> wrote:

> Your question #65246 on ScmProj plugin changed:
> https://answers.launchpad.net/bzr-scmproj/+question/65246
>
> Status: Open => Answered
>
> Alexander Belchenko proposed the following answer:
> <email address hidden> пишет:
> > New question #65246 on ScmProj plugin:
> > https://answers.launchpad.net/bzr-scmproj/+question/65246
> >
> > hi, am total newb -- to a) VCS and b) bzr...
>
> Well, some basic knowledge of bzr required here, at least you need to know
> how
> to work with separate branches.
>
> Basic thing about bzr you need to know: bzr working with branches. Every
> branch
> has repository (as internal storage for history) and could have working
> tree with
> your actual working files.
>
> Basic thing about scmproj you need to know: scmproj working with collection
> of branches
> as one big project. In the same time every branch in such project is still
> plain
> bzr branch and all typical operations applicable to every separate
> component.
> Scmproj is only provide the way to automatize some operations on the set of
> branches,
> i.e. run some command for all components at once.
>
> > am hoping you can recommend a bzr/scmproj configuration scenario for
> my use-case.
>
> If your intent is to work on set of branches then scmproj certainly will
> help you.
>
> > Drupal distributions have a core codebase,
> > plus a "multisite" domain-specific directory tree that allows one to add
> 3rd-party contributed
> > modules to extend (or override) the core Drupal distro's functionality
> --
> > **if** the file structure is hosted on a single webserver.
>
> OK, every bzr branch should have their own directory, nested branches is
> supported well
> by bzr and scmproj. You need to decide what modules/distros should go in
> which branch.
>
> > the Drupal directory tree looks like this:
> >
> > <drupal root/ (i.e. public_html/)>
> > .htaccess [frequently has server-specific
> tweaks]
> > index.php [core code]
> > .. etc.
> > modules/ [standard distro modules]
> > moduleA/
> > moduleB/
> > .. etc.
> > scripts/ [another core directory]
> > .. etc.
> > sites/ [top of Drupal's "multisite" tree]
> > all/ [added items common to all
> domains]
> > modules/ [third-party functionality
> contributions]
> > moduleX/
> > moduleY/
> > themes/ ["skins" common to all sites]
> > admin_menu/
> > domain1.com/ [domain1-specific items (i.e.
> commissioned / third-party contributions]
> > modules/
> > moduleF/
> > moduleG/
> > themes/
> > domain1_theme/ [custom theme for domain1]
> >
> > domain2.org/ [domain2-specific items (i.e.
> commissioned / third-party contributions]
> > modules/
> > moduleG/
> > moduleM/
> > themes/
> > domain2_theme/ [custom theme for domain2]
> >
> > domain3.net/ [domain3-specific items (i.e.
> commissioned / third-party contributions]
> > modules/
> > moduleN/
> > moduleO/
> > themes/
> > domain3_theme/ [custom theme for domain3]
> >
> > .. etc.
> > themes/ [core distro's themes]
> > xmlrpc.php [yet more core code]
> >
> > if all my sites were hosted on a single server, I think I'd understand
> how to set up bzr - because I'd only need one repository, and one URL to
> push to. but since I need each domain to get a) the core distribution files,
> b) the sites/all/ tree, and c) its respective domainX.TLD/ tree, and push to
> that domain's unique server URL, I'm seriously confused about the best
> approach.
> >
> > considerations:
> > Drupal.org releases minor updates to its core code distro quite
> frequently (say monthly), e.g. fixing security issues.
> > 3rd-party contributed modules are updated on independent schedules by
> their various developers. Changes can happen anywhere from annually through
> daily.
> > right now, I'm the only programmer; but I want to involve other
> developers to help me with domain-specific projects [generally writing
> domain-specific modules / themes]
>
> OK. So first at all you need to divide the big tree above into smaller
> components.
> I'd suggest make drupal core files (top-level directory) as one branch,
> and all separate components in sites/ should be separate branches.
> I'm not sure does sites/all/ should be part of Drupal core or separate
> branch
> (I'm not familiar with Drupal internals).
>
> > what I'm looking to bzr/scmproj to help me with:
>
> So again, scmproj will help to manage set of branches, can be useful to
> recreate your
> directory structure for development or deployment.
>
> > merge in site-specific development work done by me or by other
> developers
>
> Can be done as usual bzr merge for 2 branches, scmproj is not related
> here.
>
> > testing new versions of Drupal core code in a local "sandbox"
>
> Can be done with scmproj (commands project-get, project-update)
>
> > testing new versions of third-party contributions [whether they're
> shared across sites, or are site-specific] in a local sandbox
>
> Can be done with or without scmproj, because you want to inspect/testing
> one separate branch.
>
> > share the testing work for a given domain with other developers
>
> Can be done with scmproj (commands project-get to recreate project
> structure)
>
> > push a new set of core code + third-party contributions to the
> production server for a specific domain, after sandbox testing is done
>
> Can be done with scmproj (commands project-publish or project-cmd)
>
> > quickly roll back changes to a given domain (or all domains, if
> there's a problem with some code they all share), if/when problematic
> issues emerge
>
> Can be done with scmproj (command project-cmd)
>
> > if you can help steer my thinking, I'd would be very grateful!
>
> It seems doable and possible to do with scmproj.
> But your question is very big and have several subquestion inside.
> Format of Launchpad.net answers service is not very good for such things.
>
> I suggest to continue this discussion in main bzr mailing list, if you
> don't mind.
> I've cc'd this answer to bzr ML as well. You can use gmane to read and
> post.
> (http://dir.gmane.org/gmane.comp.version-control.bazaar-ng.general)
>
> >
> > best,
> >
> > Adrian Russell-Falla
>
> --
> If this answers your question, please go to the following page to let us
> know that it is solved:
>
> https://answers.launchpad.net/bzr-scmproj/+question/65246/+confirm?answer_id=0
>
> If you still need help, you can reply to this email or go to the
> following page to enter your feedback:
> https://answers.launchpad.net/bzr-scmproj/+question/65246
>
> You received this question notification because you are a direct
> subscriber of the question.
>

Revision history for this message
Alexander Belchenko (bialix) said :
#3

<email address hidden> пишет:
> so it seems like scmproj is the best answer to deal with administering *all*
> the different sites (and their respective branches) as a group -- eg. on
> getting a new Drupal distro release, to create sandboxes for each one; and
> after testing each, to to one big push up to all the different production
> servers associated with the different sites. that's very cool indeed!
>
> am going to check out the main scmproj discussion list now.

Correction: there is no "main scmproj discussion list", I've talked about
main bzr discussion list, because scmproj is bzr plugin.

I recommend for the start:

1) Install scmproj plugin (http://bialix.com/scmproj/docs/install.html)
2) Create new project in the root of your drupal tree: bzr pinit .
3) Step into new .scmproj directory, you'll see there project.cfg
4) Read about format of this config file and start edit it to describe
    your components (http://bialix.com/scmproj/docs/project-cfg.html)
    Remember that .scmproj is regular bzr branch, so you need to
    commit your changes in project.cfg when you'll make progress,
    and to allow clone project operations.

There you'll have more specific questions. Feel free to ask.

Can you help with this problem?

Provide an answer of your own, or ask adrianrf@gmail.com for more information if necessary.

To post a message you must log in.