Recognizing data-science technologies in python

[audrism]

In order to find projects relevant to learning it might be helpful to select them
from all python projects using modules specific to data science. What those may be?
How to go about finding them?

[user2589]

One way is to run community detection (e.g. Louvain algorithm) on technical dependencies network.
At the end, all ML/AI related projects fall into the same cluster (291 in the attached file)

pypi_communities.csv.gz

[jrosen48]

Perhaps we could use results from large-scale surveys?

• Kaggle's The State of Data Science and Machine Learning survey
• Stack Overflow's Developer survey

[audrism]

@jrosen48 Sure, do these surveys mention the projects used? or you intend to use surveys to develop theries for the learning research?

I think the list of ML projects in python from Marat and also R projects (we may select a subset related to ML if needed) provide a good base to determine the entire set of projects that use/learn ML.

Might it be a good idea to reconnect to see what the deadlines are and to focus ona specific set of RQs?

[jrosen48]

So, my thinking was that the surveys would possibly list specific Python libraries or R packages, but they do not appear to.

Reconnecting sounds great. Will send a note r/n.