Project

General

Profile

Bug #19678

tests: le test de SSO concurrents échoue des fois sur un 'Database is locked' de la part de sqlite

Added by Benjamin Dauvergne about 2 years ago. Updated about 1 year ago.

Status:
Fermé
Priority:
Normal
Target version:
Start date:
26 Oct 2017
Due date:
% Done:

100%

Patch proposed:
Yes
Planning:
No

Description

C'est à cause de la temporisation par défaut sur le lock entre connections qui est de 5s, des fois l'exécution des différentes requêtes prend plus de 5 secondes. Il faut augmenter la longueur de cette temporisation de façon arbitrairement grande.

0001-tests-prevent-Database-is-locked-error-during-concur.patch View (1.9 KB) Benjamin Dauvergne, 26 Oct 2017 08:55 PM

0001-tests-prevent-Database-is-locked-error-during-concur.patch View (1.94 KB) Benjamin Dauvergne, 15 Feb 2018 11:56 AM

Associated revisions

Revision f1af12e6 (diff)
Added by Benjamin Dauvergne over 1 year ago

tests: prevent "Database is locked" error during concurrency test (fixes #19678)

SQLite has a default timeout of 5 seconds, we augment it to 400 seconds. We also
replace our custom thread pool by the one provided by multiprocessing.

History

#1 Updated by Benjamin Dauvergne about 2 years ago

#2 Updated by Benjamin Dauvergne almost 2 years ago

Patch toujours ok sur master, en attente de relecture.

#3 Updated by Emmanuel Cazenave almost 2 years ago

Chez moi le patch provoque l'erreur suivante pour l'environnement pg (l'erreur ne se produit pas sur master):

test_lookup_user_transaction ______________________________________                                                                                                          

transactional_db = None, concurrency = 100                                                                                                                                                                         

    def test_lookup_user_transaction(transactional_db, concurrency):
        adapter = DefaultAdapter()
        p = ThreadPool(concurrency)

        def f(i):
            # sqlite has a default lock timeout of 5s seconds between different access to the same in
            # memory DB
            connection.cursor().execute('PRAGMA busy_timeout = 400000')
            try:
                return adapter.lookup_user(idp, saml_attributes)
            finally:
                connection.close()
>       users = p.map(f, range(concurrency))

tests/test_default_adapter.py:70:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib/python2.7/multiprocessing/pool.py:253: in map
    return self.map_async(func, iterable, chunksize).get()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <multiprocessing.pool.MapResult object at 0x7f1bdb73cc90>, timeout = None

    def get(self, timeout=None):
        self.wait(timeout)
        if not self._ready:
            raise TimeoutError
        if self._success:
            return self._value
        else:
>           raise self._value
E           ProgrammingError: syntax error at or near "PRAGMA" 
E           LINE 1: PRAGMA busy_timeout = 400000
E                   ^

Peut-être utile :

dpkg -l |grep postgres                                                                                                                                              
ii  libreoffice-sdbc-postgresql           1:5.4.4-1                           amd64        PostgreSQL SDBC driver for LibreOffice                                                                                 
ii  postgresql                            10+189                              all          object-relational SQL database (supported version)                                                                      
ii  postgresql-10                         10.1-3                              amd64        object-relational SQL database, version 10 server                                                                      
ii  postgresql-client-10                  10.1-3                              amd64        front-end programs for PostgreSQL 10                                                                                   
ii  postgresql-client-common              189                                 all          manager for multiple PostgreSQL client versions
ii  postgresql-common                     189                                 all          PostgreSQL database-cluster manager

#4 Updated by Emmanuel Cazenave almost 2 years ago

J'ai upgradé postgres (tous mon système en fait), mais ça ne change rien.

pkg -l |grep postgres
ii  libreoffice-sdbc-postgresql           1:6.0.1-1                           amd64        PostgreSQL SDBC driver for LibreOffice
ii  postgresql                            10+190                              all          object-relational SQL database (supported version)
ii  postgresql-10                         10.2-1                              amd64        object-relational SQL database, version 10 server
ii  postgresql-client-10                  10.2-1                              amd64        front-end programs for PostgreSQL 10
ii  postgresql-client-common              190                                 all          manager for multiple PostgreSQL client versions
ii  postgresql-common                     190                                 all          PostgreSQL database-cluster manager

#5 Updated by Benjamin Dauvergne almost 2 years ago

Oui ça ne concerne que sqlite, je regarde où j'ai oublié de faire un skipifpostgres.

#6 Updated by Benjamin Dauvergne almost 2 years ago

Voilà avec un if sur le connection.vendor.

#7 Updated by Emmanuel Cazenave almost 2 years ago

Ça passe, ack.

#8 Updated by Benjamin Dauvergne over 1 year ago

  • Target version set to 1.2.34

#9 Updated by Benjamin Dauvergne over 1 year ago

  • Status changed from Nouveau to Résolu (à déployer)
  • % Done changed from 0 to 100

#10 Updated by Benjamin Dauvergne about 1 year ago

  • Status changed from Résolu (à déployer) to Fermé

Also available in: Atom PDF