Skip to content

Commit

Permalink
[db] Eliminating duplicate key constraint violations
Browse files Browse the repository at this point in the history
In concurrent storage of two runs containing the same files leads to
duplicate key constraint violation. Some DBMSs can gracefully handle
this issue by supporting "ON CONFLICT DO NOTHING" clause at INSERT
statement.
  • Loading branch information
bruntib committed Jul 22, 2022
1 parent 6499dfd commit ef405cb
Showing 1 changed file with 20 additions and 2 deletions.
22 changes: 20 additions & 2 deletions web/server/codechecker_server/api/mass_store_run.py
Original file line number Diff line number Diff line change
Expand Up @@ -152,8 +152,26 @@ def add_file_record(
return file_record.id

try:
file_record = File(file_path, content_hash, None, None)
session.add(file_record)
# Parallel storage of runs containing common file paths results a
# "duplicate key violation" error. This is handled by CodeChecker, so
# practically it causes no problem. The INSERT command of the second
# transaction will be thrown away. However, some DB systems are
# supporting "ON CONFLICT DO NOTHING" clause in INSERT statement which
# solves the same issue gracefully.
# TODO: "ON CONFLICT DO NOTHING" feature is available for SQLite engine
# too in SQLAlchemy 1.4.
if session.bind.dialect.name == 'postgresql':
insert_stmt = sqlalchemy.dialects.postgresql.insert(File).values(
filepath=file_path,
filename=os.path.basename(file_path),
content_hash=content_hash).on_conflict_do_nothing(
index_elements=['id'])
file_id = session.execute(insert_stmt).inserted_primary_key[0]
session.commit()
return file_id
else:
file_record = File(file_path, content_hash, None, None)
session.add(file_record)
session.commit()
except sqlalchemy.exc.IntegrityError as ex:
LOG.error(ex)
Expand Down

0 comments on commit ef405cb

Please sign in to comment.