python3 migration

dario requested to merge of82ecuq/evaluationsportal:2to3 into master

aside from pure syntax changes, this contains two new database versions:

  • version 6 converts the timestamp datetime object to an iso format ascii string, so that python3 won't choke on it (python bug 22005)
  • version 7 is the first one to be run with python3 as interpreter, and reverts the conversion.

things to note

  • upgrade needs to happen in two steps, because db_upgrades/version6.py needs to be run with python2, but checking out the latest commit of this branch directly will contain chainges to the common db_upgrades/simple_upgrade.py, so you should check out the version6 commit (currently 0cadfd01b89e), run ./db_upgrades/version6.py, then check out the latest code (currently 2b8c4667a0) and run ./db_upgrades/version7.py. sorry for the inconvenience here.

  • i think that in general the old db_upgrades scripts should not be touched, as there's nontrivial python2-isms in there, so if ever starting from scratch again you'd be best off using the python2 tools where we did that too.

  • there's a bug still open that i'm not clear on how to fix: when downloading a pdf, the testserver throws a UnicodeDecodeError exception, somewhere in werkzeug code:

my backtrace
127.0.0.1 - - [31/Mar/2019 00:05:49] "GET /public_pdf/aafe234a539b2c71cf3c86c56ac4a332.pdf HTTP/1.1" 500 -
Error on request:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/werkzeug/serving.py", line 270, in run_wsgi
    execute(self.server.app)
  File "/usr/lib/python3/dist-packages/werkzeug/serving.py", line 260, in execute
    for data in application_iter:
  File "/usr/lib/python3/dist-packages/werkzeug/wsgi.py", line 870, in __next__
    return self._next()
  File "/usr/lib/python3/dist-packages/werkzeug/wsgi.py", line 944, in __next__
    data = self.file.read(self.buffer_size)
  File "/usr/lib/python3.7/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe4 in position 10: invalid continuation byte

Merge request reports