2024 Cohort 2 - Group 13 Discussion

[Sharon-DPC]

This is the discussion thread for those in Group 13 of the July-December 2024 Cohort

[annieukgwa]

Hi Group 13! Would you mind popping your names here in the thread or letting me know by email (I should have taken a screenshot!) and I'll use this to suggest our upcoming rota for facilitating and note taking? So the plan is before our next session to read the PSG Primer, check we have installed an instance of Python, and start on the Library Carpentry intro if we have time. And we can share any ideas or questions here in the meantime :)

[kherzy]

Annie, thanks for wrangling us into a schedule!

And already a question for the hive mind:
Both Kate and something in the reading mentioned anaconda--does anyone have experiences or opinions they'd like to share about direct download vs. anaconda? Does it matter?? Is there something special we need to do if we don't use anaconda??

Thanks,
Kyna

[annieukgwa]

Hi Kyna, no worries! I've downloaded Anaconda too - I think it's just a simple way to get started if you're using Windows, but I'm not too familiar with the pros and cons. I wonder if the Wiki/primer has any more info. I've had it suggested both on my initial code club the other year, and when I started trying this online course more recently (I didn't get beyond installing it though!) []https://melaniewalsh.github.io/Intro-Cultural-Analytics/02-Python/01-Install-Python.html

[annieukgwa]

Group 13 session rota

July: Intro session
August: Annie (facilitate), Natasha (notes)
September: Bill (facilitate), Kyna (notes)
October: Doug (facilitate), Kate (notes)
November: Kate (facilitate), Doug (notes)
December: Kyna (facilitate), Bill (notes)

Hope that looks ok to everyone! We can of course rejig depending on availability and how we find it goes. Perhaps for the next session we could start by sharing how we've all got on between sessions, discuss any problems, then take some study time to work through some more beginner exercises - either together or separately - and maybe regroup at the end to discuss our plan for next time. Other format ideas very welcome!

[wlowry]

A guide to Python for audio visual tasks: https://amiaopensource.github.io/av-python-carpentry/index.html

[annieukgwa]

A few different intro courses I'm aware of or have attempted to start at various points in time:

Library Carpentry
Automate The Boring Stuff
Introduction to Cultural Analytics & Python

[alexwlchan]

Lovely to meet you all today!

Here's the example code we wrote today using break and continue:

for number in [1, 2, 3, 4, 5, 6, 7, 8]:
    if number == 5:
        continue
    if number > 6:
        break
    print(number)
    break

[annieukgwa]

Brilliant! Thank you so much for sharing your Python wisdom in today's session, Alex - it was really helpful and encouraging :)

[kate-lib]

Thank you!

[wlowry]

Python Study Group Meeting - 21st August 2024

Annie (facilitate), Bill (notes)
Apologies: Natasha, Doug

There were no general announcements, we went straight into our breakout rooms.

Everyone gave an update of their progress over the last month:
• Kate has been using Library Carpentry (https://librarycarpentry.org/library-python/)
• Kyna has been working with Visual Studio (https://visualstudio.microsoft.com/) as an editor and has also making her way through the Primer (https://github.com/Digital-Preservation-Coalition/PythonStudyGroups/wiki).
• Annie has been working with Anaconda, Jupyter Notebooks, Automate the Boring Stuff (https://automatetheboringstuff.com/) and Library Carpentry.
• Bill has been working through the “How To’s”.

The group was joined by Alex Chan as a mentor. Alex spoke about the environments that some of the group have been using such as Anaconda and also spoke about Spyder (https://www.spyder-ide.org/) which is often used for data science. Alex also mentioned the alternative ways that Python can be installed such as “System Python” (this is typically pre-installed on many operating systems) and installing direct from the Python website. Alex often runs Python in virtual environments so that they can use different versions that don’t interact with each other.

Kate raised concerns that Anaconda was trying to “help” too much and that this would make the work too easy and that she might miss fundamental parts of learning. Alex replied that Anaconda has built in guardrails to stop you making mistakes.
Alex spoke about the Spyder environment and mentioned REPL (Read Eval Print Loop). Alex also spoke about Jupyter and pointed out some features and showed how to include notes around the code.
Kyna asked about running the code in development environments and whether it could affect real files (modifying, deleting etc). Alex stated that it probably would (so we should be careful)!
Alex demonstrated Loops and showed how code repeated itself to run code over and over to use new variables. They also showed the IF and BREAK keywords. IF allows a section code to run if a condition is True. BREAK simply stops the code running and is useful if you’re testing and only want to run the code until that point.
Next meeting: 18th September 2024 - Bill (facilitate), Kyna (notes)

[kate-lib]

Python Study Group Meeting - 16th October 2024

Doug (facilitate), Kate (notes)
Apologies: Bill

There were no general announcements, we went straight into our breakout rooms.

Everyone gave an update of their progress:

• Annie is working through coursebook, thinking about how to apply theoretical knowledge to real-life work. Her web archiving team has been reviewing code that ChatGPT has suggested. Hoping to learn how to use Python to monitor about 500 dormant websites for changes in case they need to be crawled again.

• Doug has 700 Bags of digitized content to validate, trying to find a script that can do it. Found a script online but still trying to get it to run. But then discovered that “bag.py—validate” seems to work! Now wants to find a way to generate a report.

• Kate is trying to learn about web scraping to download large numbers of PDF documents from a webpage. Playing with some scripts from Stack Overflow and GeeksforGeeks, after some trial and error got the code to run.

One of the mentors, Harry Lloyd, popped in, although we didn’t have any questions for him at the time (thank you anyway, Harry!) We used the rest of the time for independent study.

Next meeting: 20th November 2024 - Kate (facilitate), Doug (notes)

[kate-lib]

Code to scrape PDFs from webpage

Hi everyone, I was working on this last time, just sharing here in case anyone else might find it useful. This code finds PDF files hosted on a particular webpage and downloads them.

Adapted from https://www.geeksforgeeks.org/downloading-pdfs-with-python-using-requests-and-beautifulsoup/ and from https://stackoverflow.com/questions/54616638/download-all-pdf-files-from-a-website-using-python

#Import os, requests, BeautifulSoup, and urljoin:
#os provides functions for interacting with the operating system
#requests allows you to send http requests
#BeautifulSoup (4) is a Python library for pulling data out of HTML and XML files
#urllib.parse is a module for separating or rejoining components of URLs
#urljoin lets you construct a full (“absolute”) URL by combining a “base URL” (base) with another URL (url).

import os
import requests
from bs4 import BeautifulSoup
from urllib.parse import urljoin

#Request URL and get response object
url = "https://www.yourwebsiteaddress.com"
response = requests.get(url)

#look through html webpage to find links with tag "a href" that end in .pdf
soup= BeautifulSoup(response.text, "html.parser")

for link in soup.select("a[href$='.pdf']"):
#For every link found, names the pdf file using the last portion of each link which is unique
filename = os.path.join(link['href'].split('/')[-1])
#Downloads the PDF file
with open(filename, 'wb') as f:
f.write(requests.get(urljoin(url,link['href'])).content)

[wlowry]

Python study Group Meeting - 18th December 2024

Attending: Bill, Kyna, Annie
Apologies: Kate, Doug, Natasha

There was a brief presentation from Amy at the start of the meeting (Slides). The main takeaways were that the GitHub repo will remain up and Study Groups are free to continue meeting but will need to host their own sessions.
Feedback forms will be sent out in the new year.

The group gave individual updates and we decided to keep an email chain going so that we can bounce ideas off each other in the future. It was felt that email was preferable to the GitHub page which was rarely used and is public.
If anyone wants to opt out of the email chain please let us know.