Skip to content

Commit

Permalink
Cleaning database if history not matching DB
Browse files Browse the repository at this point in the history
This update brings improvements to the multiple scrobbles of a song functionality.

Script now compares the records returned from the YT API to the entries in the local DB. If a record exists in the DB but not in the API response it will be deleted from the DB.

We can assume that since we filter on "Today" that if a record does not exist at time x but then at future time y; it's a new scrobble.

This also keeps the local DB nice and tidy/performance improvements.

It also makes the script work regardless of time-zone of the user since we only modify the local DB based on the API response now.

Sidenote for people looking at this script for educational purpouses: Note that constructing the DELETE statement dynamically can be risky with a large number of records due to SQL injection risks and query length limits. In a production environment, you'd want to handle this more robustly, perhaps by using a temporary table or other safe means. Since we know this will run locally and we are not dealing with enterprise-size clusters of data here I did the query the quick and dirty way.
  • Loading branch information
roos-robert committed Jun 15, 2024
1 parent ca1f0fa commit 602fa92
Showing 1 changed file with 35 additions and 0 deletions.
35 changes: 35 additions & 0 deletions start.py
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,41 @@ def execute(self):
history = ytmusic.get_history()
i = 0
cursor = self.conn.cursor()

# Performing scrobbling of songs, cleaning up local DB if needed first, then adding/updating records to local DB and Last.FM
# Step 1: Collect all today's scrobbles
today_records = []

for index, item in enumerate(history):
if item["played"] == "Today":
record = {
"artistName": item["artists"][0]["name"],
"trackName": item["title"],
"ts": self.formatted_date,
"albumName": item["album"]["name"] if "album" in item and item["album"] is not None else None,
"arrayPosition": index,
}
if record["artistName"].endswith(" - Topic"):
continue
if record["albumName"] is None:
record["albumName"] = record["trackName"]

today_records.append((record["trackName"], record["artistName"], record["albumName"]))

# Step 2: Delete all records in the database that are not in today's scrobbles
if today_records:
placeholders = ', '.join(['(?, ?, ?)'] * len(today_records))
flat_today_records = [item for sublist in today_records for item in sublist]
cursor.execute(f'''
DELETE FROM scrobbles
WHERE (track_name, artist_name, album_name) NOT IN (
{placeholders}
)
''', flat_today_records)
self.conn.commit()

# Step 3: Insert or update today's scrobbles
i = 0 # Initialize i for scrobble timing
for index, item in enumerate(history):
if item["played"] == "Today":
record = {
Expand Down

0 comments on commit 602fa92

Please sign in to comment.