Using migrate.py and debugging messages [6.1-7.0]

you wouldn't have to download anything besides the script, it does that for you. But there's no problem with the way you did it either.

Now the scripts gives two errors which seem to be fatal, and the error log you post shows that the migration stopped in a very early stage if it ran at all.

The script writes the error log to $branch-dir/migration.log, could you post that one please?

Alexandro, thanks for creating a new issue!

The stdout of the database restore shows severe problems

The first error seems to be key:

pg_restore: [archiver (db)] COPY failed for table "res_currency": ERROR: value too long for type character varying(3)
CONTEXT: COPY res_currency, line 27, column symbol: "лв"

Looks like your template1 database is not UTF-8. Apparently, the script does not enforce this either. Try adding WITH ENCODING 'UTF8' to the query in line 192 of migrate.py.

@Holger: hi, this is the migration.log content:
http://pastebin.mozilla.org/3769266

@Stefan:
I see, I have added the 'With encoding 'utf8', template1 was created as SQL_ASCII instead of UTF8 even thought the 6.1 db is UTF8 will report back on this.

the log you just posted also points to encoding problems, so keep us updated if Stefan's suggestion solves the problem

I was able to get the script running now I am presented with the following path:
/home/user/tmp/openupgrade/7.0/
 ...server/scripts/
 ...web/
 ...addons/
 ...server.cfg

I ran the server/openerp-server --xmlrpc-port=1800 and showed some errors:

ImportError: No module named web
2013-12-13 17:11:35,329 20656 WARNING ? openerp.modules.module: module web: module not found
2013-12-13 17:11:35,331 20656 CRITICAL ? openerp.modules.module: Couldn't load module web
2013-12-13 17:11:35,331 20656 CRITICAL ? openerp.modules.module: No module named web
2013-12-13 17:11:35,331 20656 ERROR ? openerp.service: Failed to load server-wide module `web`.
The `web` module is provided by the addons found in the `openerp-web` project.
Maybe you forgot to add those addons in your addons_path configuration.

Here is the second error:
ImportError: No module named web
2013-12-13 17:11:35,332 20656 WARNING ? openerp.modules.module: module web_kanban: module not found
2013-12-13 17:11:35,333 20656 CRITICAL ? openerp.modules.module: Couldn't load module web_kanban
2013-12-13 17:11:35,333 20656 CRITICAL ? openerp.modules.module: No module named web_kanban
2013-12-13 17:11:35,334 20656 ERROR ? openerp.service: Failed to load server-wide module `web_kanban`.

so you ran migrate.py, it didn't show any errors and now you try to run openerp afterwards?

if this is the case, you're done already. Consider everything he created in tmp/openupgrade as temporary files. just run your regular 7.0 installation and use the _migrated version of your database.

...if there were any issues, they should have been logged in migration.log, so please keep that file and post it in case of further questions

I did experience many errors:

2013-12-13 19:16:22,044 26645 ERROR prueba1 openerp.modules.graph: module sale_crm: Unmet dependencies: crm
2013-12-13 19:16:22,044 26645 ERROR prueba1 openerp.modules.graph: module project: Unmet dependencies: base_status
2013-12-13 19:16:22,044 26645 ERROR prueba1 openerp.modules.graph: module sale_mrp: Unmet dependencies: sale_stock
2013-12-13 19:16:22,048 26645 ERROR prueba1 openerp.modules.graph: module base_calendar: Unmet dependencies: base_status
2013-12-13 19:16:22,049 26645 ERROR prueba1 openerp.modules.graph: module crm: Unmet dependencies: base_status, base_calendar

and some warning:

2013-12-13 19:16:21,858 26645 WARNING prueba1 openerp.modules.module: module base_tools: module not found
2013-12-13 19:16:21,869 26645 WARNING prueba1 openerp.modules.module: module base_tools: module not found
2013-12-13 19:16:21,869 26645 WARNING prueba1 openerp.modules.module: module base_tools: module not found
2013-12-13 19:16:21,870 26645 WARNING prueba1 openerp.modules.graph: module base_tools: not installable, skipped
2013-12-13 19:16:21,890 26645 WARNING prueba1 openerp.modules.module: module web_process: module not found
2013-12-13 19:16:21,894 26645 WARNING prueba1 openerp.modules.module: module web_process: module not found
2013-12-13 19:16:21,895 26645 WARNING prueba1 openerp.modules.module: module web_process: module not found
2013-12-13 19:16:21,899 26645 WARNING prueba1 openerp.modules.graph: module web_process: not installable, skipped
2013-12-13 19:16:21,935 26645 WARNING prueba1 openerp.modules.module: module web_mobile: module not found
2013-12-13 19:16:21,936 26645 WARNING prueba1 openerp.modules.module: module web_mobile: module not found
2013-12-13 19:16:21,936 26645 WARNING prueba1 openerp.modules.module: module web_mobile: module not found
2013-12-13 19:16:21,936 26645 WARNING prueba1 openerp.modules.graph: module web_mobile: not installable, skipped
2013-12-13 19:16:21,985 26645 WARNING prueba1 openerp.modules.module: module fetchmail_crm: module not found
2013-12-13 19:16:21,986 26645 WARNING prueba1 openerp.modules.module: module fetchmail_crm: module not found
2013-12-13 19:16:21,987 26645 WARNING prueba1 openerp.modules.module: module fetchmail_crm: module not found
2013-12-13 19:16:21,987 26645 WARNING prueba1 openerp.modules.graph: module fetchmail_crm: not installable, skipped
2013-12-13 19:16:21,987 26645 WARNING prueba1 openerp.modules.module: module web_dashboard: module not found
2013-12-13 19:16:21,989 26645 WARNING prueba1 openerp.modules.module: module web_dashboard: module not found
2013-12-13 19:16:21,995 26645 WARNING prueba1 openerp.modules.module: module web_dashboard: module not found
2013-12-13 19:16:21,996 26645 WARNING prueba1 openerp.modules.graph: module web_dashboard: not installable, skipped

The migration.log got quite large, but I paste many of the messages in it.
http://pastebin.mozilla.org/3770192

No, that did not go well. I gather from your last comment that this is not the complete log, is it? We do need that to say something more.

Here it goes:

http://pastebin.mozilla.org/3770301

  File "base/migrations/7.0.1.3/pre-migration.py", line 155, in create_users_partner
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2: ordinal not in range(128)

So more encoding woes. Not sure which revision you're on, but that method only contains database cursor interactions. As OpenERP sets psycopg2.extensions.UNICODE, this should not happen on a system with correctly configured encodings. You should check all of that.

where can I look around for 'ascii' type encoding, from the postgresql channel they told me to check the content of the variables from the database. I can see that the errors were not only on res_users but also many other tables were affected, however the final daatabase do show the same ammount of data on both res_user table and res_partner.

Can you help me trying to see which data is causing the errors?

Look at "base/migrations/7.0.1.3/pre-migration.py", line 155. My guess is that this is where partner names are selected. The 'offensive' data is probably partners with diacritics in their names, such as 'éàü'.

Just curious: have you ever worked with a backup copy of your database? Did you experience anything like this?

Tried a new database, and this time this gave me a different output.
http://pastebin.mozilla.org/3794692

I didnt had any error regarding encoding this time, however some more mild errors, but I stil have no idea how to go about them. I assume that the TZ filed was the one defined at the configuration, and not setting it up made the migration script somewhat confused. At least that's what I think it happened.

The modules that were affected the most were:
res.company = * res.company.form.inherit
res.users = * res.users.form

All the errors complain about not being able to find:
context_tz, menu_tips, context_lang
and from res.company, the field property_reserve_and_surplus_account and security_lead

@Stefan
Thanks,
Seems the UTF8 issues are solved, but I am still experiencing difficulties, this time look more migration-proper issue, regarding the res.users/company. I've heard there were changes to the layout of these fields between 6.1 and 7.0.

The script seems to have coughed them out and I am still not able to login on the 7.0 without experriencing the same issue I paste on comment.

ProgrammingError: column res_users.alias_id does not exist
LINE 1: ...CT res_users."menu_id",res_users."gmail_password",res_users....

Hi Alexandro,

the errors about missing fields in views are expected, so that's okay.

Your log from december 17th looks okay so far, but it stops in the middle at line 522. Any chance you could upload the complete log?

Further just to be sure: You do your login on database astro_migrated and not the original one, right?

Could you verify with
psql astro_migrated
\d res_users
that the alias_id column is created after the migration?

hi I think I am out of the woods with this, but still have issues on a post-migration level now. BTW the issue was that the migration was not even generating the data information on the migrated table. Mainly because the binary was failing to import back into the postgresql.

At the moment it seems some of the models are just having issues on the new platform. Another issue I found is on the views, for example I am unable to watch my invoices (on accounting -> client/partners -> invoices). The form view throws an XML Attribute error:
AttributeError: View definition error for inherited view 'account_invoice_layout.account_invoice_form_inherit_1' on model 'account.invoice': Element '<xpath expr="/form/notebook/page/field[@name='invoice_line']">' not found in parent view 'account.invoice_form'

Mainly the accounting.invoice depends on a parent view: http://pastebin.mozilla.org/3917962 and an inherit view http://pastebin.mozilla.org/3917794

I access the views on Settings -> Technical -> User Interface -> Views -> account.invoice.form / account_invoice_layout.account_invoice_form_inherit_1
Here is a screenshot: http://imagebin.org/284799

The error seems to happen around line 53 of the account.invoice.form.

good to hear. So I conclude your problems were caused by some database setup issues? If you feel that those are problems other people might run into too, it would be great if you could do a summary of what went wrong and how you fixed it.

account_invoice_layout is not supported in version 7.0 any more, so you'll have to deinstall this module (or port it to version 7.0): http://v6.openerp.com/node/1269#h.k5rq39nvqtb4

Oh thank you for telling me. I have been trying to fix this for a few days now and I was going a bit crazy trying to solve it. I had similar issues with other modules like web mobile addon which also has been deprecated in version 7.0.

So my only issue here is how to finalize the migration. Meaning, after I uninstall the account_invoice_layout, how will the client be able to pull out his invoices (under Accounting -> Client -> Client Invoice).

Will uninstalling this and run an update would be all that is needed or should I install something on its place manually?

Regards, and Happy new year.

Wow, I went ahead an uninstalled the module account_invoice_layout and now I am presented with the following, more serious bug report:

" File "/home/ubuntu/openerp7/openerp/osv/osv.py", line 185, in execute_cr
    return getattr(object, method)(cr, uid, *args, **kw)
  File "/home/ubuntu/openerp7/openerp/osv/orm.py", line 3623, in read
    result = self._read_flat(cr, user, select, fields, context, load)
  File "/home/ubuntu/openerp7/openerp/osv/orm.py", line 3744, in _read_flat
    res2 = self._columns[f].get(cr, self, ids, f, user, context=context, values=res)
  File "/home/ubuntu/openerp7/openerp/osv/fields.py", line 538, in get
    ids2 = obj.pool.get(self._obj).search(cr, user, domain + [(self._fields_id, 'in', ids)], limit=self._limit, context=context)
  File "/home/ubuntu/openerp7/openerp/osv/orm.py", line 2369, in search
    return self._search(cr, user, args, offset=offset, limit=limit, order=order, context=context, count=count)
  File "/home/ubuntu/openerp7/openerp/osv/orm.py", line 4887, in _search
    cr.execute('SELECT "%s".id FROM ' % self._table + from_clause + where_str + order_by + limit_str + offset_str, where_clause_params)
  File "/home/ubuntu/openerp7/openerp/sql_db.py", line 161, in wrapper
    return f(self, *args, **kwargs)
  File "/home/ubuntu/openerp7/openerp/sql_db.py", line 226, in execute
    res = self._obj.execute(query, params)
ProgrammingError: column account_invoice_line.sequence does not exist
LINE 1: ...ORDER BY "account_invoice_line__invoice_id"."id" ,"account_i..."

Seems this is the only most important issue I am having to get the 7.0 operational. Please help.

after perfoming another --update all, the table was updated and I can see the client invoices. Sorry for the overreaction. I am closing this ticket.

Regards.