Skip to content

Latest commit

 

History

History
382 lines (342 loc) · 41.8 KB

awesome-python-machine-learning.md

File metadata and controls

382 lines (342 loc) · 41.8 KB
Raw

Awesome Python Machine Learning Awesome

A curated list of awesome Python frameworks, libraries, software and resources for Machine Learning.

Inspired/Forked from awesome-python, awesome-machine-learning.

Interactive Interpreter

Interactive Python interpreters (REPL).

Running your own Jupyter Notebook online (free for 2 months)

Use One-click "Machine Learning and AI" image on DigitalOcean to play around with Python using Jupyter Notebook. Use my referral link and receive $10 in credit immediately as new user, enough to run a small 1GB-RAM 25GB-SSD server for 2 months, $5 a month after. Let me know if you need help with the setup. https://m.do.co/t/b298d6966c0c I get some free credit (not cash) to run my server there if you sign up using the link.

Deep Learning

Frameworks for Neural Networks and Deep Learning. See: awesome-deep-learning.

  • Keras - A high-level neural networks library and capable of running on top of either TensorFlow or Theano.
  • TensorFlow - The most popular Deep Learning framework created by Google.
  • Caffe - A fast open framework for deep learning..
  • MXNet - A deep learning framework designed for both efficiency and flexibility.
  • Neupy - Running and testing different Artificial Neural Networks algorithms.
  • Pytorch - Tensors and Dynamic neural networks in Python with strong GPU acceleration.
  • Serpent.AI - Game agent framework. Use any video game as a deep learning sandbox.
  • Theano - A library for fast numerical computation.

Machine Learning

Libraries for Machine Learning. See: awesome-machine-learning.

Natural Language Processing

Libraries for working with human languages.

  • gensim - Topic Modelling for Humans.
  • Jieba - Chinese text segmentation.
  • langid.py - Stand-alone language identification system.
  • NLTK - A leading platform for building Python programs to work with human language data.
  • Pattern - A web mining module for the Python.
  • polyglot - Natural language pipeline supporting hundreds of languages.
  • SnowNLP - A library for processing Chinese text.
  • spaCy - A library for industrial-strength natural language processing in Python and Cython.
  • TextBlob - Providing a consistent API for diving into common NLP tasks.
  • Quepy - A python framework to transform natural language questions to queries in a database query language.
  • YAlign - A sentence aligner, a friendly tool for extracting parallel sentences from comparable corpora.
  • spammy - A library for email Spam filtering built on top of nltk
  • loso - Another Chinese segmentation library.
  • genius - A Chinese segment base on Conditional Random Field.
  • KoNLPy - A Python package for Korean natural language processing.
  • nut - Natural language Understanding Toolkit.
  • Rosetta - Text processing tools and wrappers (e.g. Vowpal Wabbit)
  • BLLIP Parser - Python bindings for the BLLIP Natural Language Parser (also known as the Charniak-Johnson parser).
  • PyNLPl - Python Natural Language Processing Library. General purpose NLP library for Python. Also contains some specific modules for parsing common NLP formats, most notably for FoLiA, but also ARPA language models, Moses phrasetables, GIZA++ alignments.
  • python-ucto - Python binding to ucto (a unicode-aware rule-based tokenizer for various languages).
  • python-frog - Python binding to Frog, an NLP suite for Dutch. (pos tagging, lemmatisation, dependency parsing, NER)
  • python-zpar - Python bindings for ZPar, a statistical part-of-speech-tagger, constiuency parser, and dependency parser for English.
  • colibri-core - Python binding to C++ library for extracting and working with with basic linguistic constructions such as n-grams and skipgrams in a quick and memory-efficient way.
  • PyStanfordDependencies - Python interface for converting Penn Treebank trees to Stanford Dependencies.
  • Distance - Levenshtein and Hamming distance computation.
  • Fuzzy Wuzzy - Fuzzy String Matching in Python.
  • jellyfish - a python library for doing approximate and phonetic matching of strings.
  • editdistance - fast implementation of edit distance.
  • textacy - higher-level NLP built on Spacy.
  • stanford-corenlp-python - Python wrapper for Stanford CoreNLP
  • CLTK - The Classical Language Toolkit.
  • rasa_nlu - turn natural language into structured data.
  • yase - Transcode sentence (or other sequence) to list of word vector .
  • DrQA - Reading Wikipedia to answer open-domain questions.
  • Dedupe - A python library for accurate and scaleable fuzzy matching, record deduplication and entity-resolution.

Text Processing

Libraries for parsing and manipulating plain texts.

  • General
    • chardet - Python 2/3 compatible character encoding detector.
    • difflib - (Python standard library) Helpers for computing deltas.
    • ftfy - Makes Unicode text less broken and more consistent automagically.
    • fuzzywuzzy - Fuzzy String Matching.
    • Levenshtein - Fast computation of Levenshtein distance and string similarity.
    • pangu.py - Spacing texts for CJK and alphanumerics.
    • pyfiglet - An implementation of figlet written in Python.
    • pypinyin - Convert Chinese hanzi to pinyin.
    • shortuuid - A generator library for concise, unambiguous and URL-safe UUIDs.
    • unidecode - ASCII transliterations of Unicode text.
    • uniout - Print readable chars instead of the escaped string.
    • xpinyin - A library to translate Chinese hanzi (漢字) to pinyin (拼音).
  • Slugify
    • awesome-slugify - A Python slugify library that can preserve unicode.
    • python-slugify - A Python slugify library that translates unicode to ASCII.
    • unicode-slugify - A slugifier that generates unicode slugs with Django as a dependency.
  • Parser
    • phonenumbers - Parsing, formatting, storing and validating international phone numbers.
    • PLY - Implementation of lex and yacc parsing tools for Python
    • Pygments - A generic syntax highlighter.
    • pyparsing - A general purpose framework for generating parsers.
    • python-nameparser - Parsing human names into their individual components.
    • python-user-agents - Browser user agent parser.
    • sqlparse - A non-validating SQL parser.

Web Content Extracting

Libraries for extracting web contents.

  • Haul - An Extensible Image Crawler.
  • html2text - Convert HTML to Markdown-formatted text.
  • lassie - Web Content Retrieval for Humans.
  • micawber - A small library for extracting rich content from URLs.
  • newspaper - News extraction, article extraction and content curation in Python.
  • python-goose - HTML Content/Article Extractor.
  • python-readability - Fast Python port of arc90's readability tool.
  • requests-html - Pythonic HTML Parsing for Humans.
  • sanitize - Bringing sanity to world of messed-up data.
  • sumy - A module for automatic summarization of text documents and HTML pages.
  • textract - Extract text from any document, Word, PowerPoint, PDFs, etc.
  • toapi - Every web site provides APIs.

Web Crawling

Libraries for scraping websites.

  • cola - A distributed crawling framework.
  • Demiurge - PyQuery-based scraping micro-framework.
  • feedparser - Universal feed parser.
  • Grab - Site scraping framework.
  • MechanicalSoup - A Python library for automating interaction with websites.
  • portia - Visual scraping for Scrapy.
  • pyspider - A powerful spider system.
  • RoboBrowser - A simple, Pythonic library for browsing the web without a standalone web browser.
  • Scrapy - A fast high-level screen scraping and web crawling framework.

Algorithms and Design Patterns

Python implementation of algorithms and design patterns.

  • algorithms - A module of algorithms for Python.
  • PyPattyrn - A simple yet effective library for implementing common design patterns.
  • python-patterns - A collection of design patterns in Python.
  • sortedcontainers - Fast, pure-Python implementation of SortedList, SortedDict, and SortedSet types.
  • SCOOP - Scalable Concurrent Operations in Python

Cryptography

  • cryptography - A package designed to expose cryptographic primitives and recipes to Python developers.
  • hashids - Implementation of hashids in Python.
  • Paramiko - A Python (2.6+, 3.3+) implementation of the SSHv2 protocol, providing both client and server functionality.
  • Passlib - Secure password storage/hashing library, very high level.
  • PyNacl - Python binding to the Networking and Cryptography (NaCl) library.

Data Analysis

Libraries for data analyzing.

  • NumPy - A fundamental package for scientific computing with Python.
  • SciPy - A Python-based ecosystem of open-source software for mathematics, science, and engineering.
  • Blaze - NumPy and Pandas interface to Big Data.
  • Open Mining - Business Intelligence (BI) in Pandas interface.
  • Orange - Data mining, data visualization, analysis and machine learning through visual programming or scripts.
  • Pandas - A library providing high-performance, easy-to-use data structures and data analysis tools.
  • Numba - Python JIT (just in time) complier to LLVM aimed at scientific Python by the developers of Cython and NumPy.
  • NetworkX - A high-productivity software for complex networks.
  • igraph - binding to igraph library - General purpose graph library.
  • Pandas - A library providing high-performance, easy-to-use data structures and data analysis tools.
  • Open Mining - Business Intelligence (BI) in Python (Pandas web interface)
  • PyMC - Markov Chain Monte Carlo sampling toolkit.
  • zipline - A Pythonic algorithmic trading library.
  • PyDy - Short for Python Dynamics, used to assist with workflow in the modeling of dynamic motion based around NumPy, SciPy, IPython, and matplotlib.
  • SymPy - A Python library for symbolic mathematics.
  • statsmodels - Statistical modeling and econometrics in Python.
  • astropy - A community Python library for Astronomy.
  • matplotlib - A Python 2D plotting library.
  • bokeh - Interactive Web Plotting for Python.
  • plotly - Collaborative web plotting for Python and matplotlib.
  • vincent - A Python to Vega translator.
  • d3py - A plotting library for Python, based on D3.js.
  • PyDexter - Simple plotting for Python. Wrapper for D3xterjs; easily render charts in-browser.
  • ggplot - Same API as ggplot2 for R.
  • ggfortify - Unified interface to ggplot2 popular R packages.
  • Kartograph.py - Rendering beautiful SVG maps in Python.
  • pygal - A Python SVG Charts Creator.
  • PyQtGraph - A pure-python graphics and GUI library built on PyQt4 / PySide and NumPy.
  • pycascading
  • Petrel - Tools for writing, submitting, debugging, and monitoring Storm topologies in pure Python.
  • Blaze - NumPy and Pandas interface to Big Data.
  • emcee - The Python ensemble sampling toolkit for affine-invariant MCMC.
  • windML - A Python Framework for Wind Energy Analysis and Prediction.
  • vispy - GPU-based high-performance interactive OpenGL 2D/3D data visualization library.
  • cerebro2 A web-based visualization and debugging platform for NuPIC.
  • NuPIC Studio An all-in-one NuPIC Hierarchical Temporal Memory visualization and debugging super-tool!
  • SparklingPandas Pandas on PySpark (POPS).
  • Seaborn - A python visualization library based on matplotlib.
  • bqplot - An API for plotting in Jupyter (IPython).
  • pastalog - Simple, realtime visualization of neural network training performance.
  • caravel - A data exploration platform designed to be visual, intuitive, and interactive.
  • Dora - Tools for exploratory data analysis in Python.
  • Ruffus - Computation Pipeline library for python.
  • SOMPY - Self Organizing Map written in Python (Uses neural networks for data analysis).
  • somoclu Massively parallel self-organizing maps: accelerate training on multicore CPUs, GPUs, and clusters, has python API.
  • HDBScan - implementation of the hdbscan algorithm in Python - used for clustering
  • visualize_ML - A python package for data exploration and data analysis.
  • scikit-plot - A visualization library for quick and easy generation of common plots in data analysis and machine learning.
  • Bowtie - A dashboard library for interactive visualizations using flask socketio and react.
  • lime - Lime is about explaining what machine learning classifiers (or models) are doing. It is able to explain any black box classifier, with two or more classes.

Computer Vision

  • Scikit-Image - A collection of algorithms for image processing in Python.
  • SimpleCV - An open source computer vision framework that gives access to several high-powered computer vision libraries, such as OpenCV. Written on Python and runs on Mac, Windows, and Ubuntu Linux.
  • Vigranumpy - Python bindings for the VIGRA C++ computer vision library.
  • OpenFace - Free and open source face recognition with deep neural networks.
  • PCV - Open source Python module for computer vision.
  • face_recognition - Face recognition library that recognize and manipulate faces from Python or from the command line.
  • dockerface - Easy to install and use deep learning Faster R-CNN face detection for images and video in a docker container.
  • Detectron - FAIR's software system that implements state-of-the-art object detection algorithms, including Mask R-CNN. It is written in Python and powered by the Caffe2 deep learning framework.

General-Purpose Machine Learning

  • CNTK - Microsoft Cognitive Toolkit (CNTK), an open source deep-learning toolkit. Documentation can be found here.
  • auto_ml - Automated machine learning for production and analytics. Lets you focus on the fun parts of ML, while outputting production-ready code, and detailed analytics of your dataset and results. Includes support for NLP, XGBoost, CatBoost, LightGBM, and soon, deep learning.
  • machine learning - automated build consisting of a web-interface, and set of programmatic-interface API, for support vector machines. Corresponding dataset(s) are stored into a SQL database, then generated model(s) used for prediction(s), are stored into a NoSQL datastore.
  • XGBoost - Python bindings for eXtreme Gradient Boosting (Tree) Library.
  • Bayesian Methods for Hackers - Book/iPython notebooks on Probabilistic Programming in Python.
  • Featureforge A set of tools for creating and testing machine learning features, with a scikit-learn compatible API.
  • MLlib in Apache Spark - Distributed machine learning library in Spark
  • Hydrosphere Mist - a service for deployment Apache Spark MLLib machine learning models as realtime, batch or reactive web services.
  • scikit-learn - A Python module for machine learning built on top of SciPy.
  • metric-learn - A Python module for metric learning.
  • SimpleAI Python implementation of many of the artificial intelligence algorithms described on the book "Artificial Intelligence, a Modern Approach". It focuses on providing an easy to use, well documented and tested library.
  • astroML - Machine Learning and Data Mining for Astronomy.
  • graphlab-create - A library with various machine learning models (regression, clustering, recommender systems, graph analytics, etc.) implemented on top of a disk-backed DataFrame.
  • BigML - A library that contacts external servers.
  • pattern - Web mining module for Python.
  • NuPIC - Numenta Platform for Intelligent Computing.
  • Pylearn2 - A Machine Learning library based on Theano.
  • keras - Modular neural network library based on Theano.
  • Lasagne - Lightweight library to build and train neural networks in Theano.
  • hebel - GPU-Accelerated Deep Learning Library in Python.
  • Chainer - Flexible neural network framework.
  • prophet - Fast and automated time series forecasting framework by Facebook.
  • gensim - Topic Modelling for Humans.
  • topik - Topic modelling toolkit.
  • PyBrain - Another Python Machine Learning Library.
  • Brainstorm - Fast, flexible and fun neural networks. This is the successor of PyBrain.
  • Surprise - A scikit for building and analyzing recommender systems.
  • Crab - A flexible, fast recommender engine.
  • python-recsys - A Python library for implementing a Recommender System.
  • thinking bayes - Book on Bayesian Analysis.
  • Image-to-Image Translation with Conditional Adversarial Networks - Implementation of image to image (pix2pix) translation from the paper by isola et al.[DEEP LEARNING]
  • Restricted Boltzmann Machines -Restricted Boltzmann Machines in Python. [DEEP LEARNING]
  • Bolt - Bolt Online Learning Toolbox.
  • CoverTree - Python implementation of cover trees, near-drop-in replacement for scipy.spatial.kdtree
  • nilearn - Machine learning for NeuroImaging in Python.
  • neuropredict - Aimed at novice machine learners and non-expert programmers, this package offers easy (no coding needed) and comprehensive machine learning (evaluation and full report of predictive performance WITHOUT requiring you to code) in Python for NeuroImaging and any other type of features. This is aimed at absorbing the much of the ML workflow, unlike other packages like nilearn and pymvpa, which require you to learn their API and code to produce anything useful.
  • imbalanced-learn - Python module to perform under sampling and over sampling with various techniques.
  • Shogun - The Shogun Machine Learning Toolbox.
  • Pyevolve - Genetic algorithm framework.
  • Caffe - A deep learning framework developed with cleanliness, readability, and speed in mind.
  • breze - Theano based library for deep and recurrent neural networks.
  • pyhsmm - library for approximate unsupervised inference in Bayesian Hidden Markov Models (HMMs) and explicit-duration Hidden semi-Markov Models (HSMMs), focusing on the Bayesian Nonparametric extensions, the HDP-HMM and HDP-HSMM, mostly with weak-limit approximations.
  • mrjob - A library to let Python program run on Hadoop.
  • SKLL - A wrapper around scikit-learn that makes it simpler to conduct experiments.
  • neurolab - https://github.com/zueve/neurolab
  • Spearmint - Spearmint is a package to perform Bayesian optimization according to the algorithms outlined in the paper: Practical Bayesian Optimization of Machine Learning Algorithms. Jasper Snoek, Hugo Larochelle and Ryan P. Adams. Advances in Neural Information Processing Systems, 2012.
  • Pebl - Python Environment for Bayesian Learning.
  • Theano - Optimizing GPU-meta-programming code generating array oriented optimizing math compiler in Python.
  • TensorFlow - Open source software library for numerical computation using data flow graphs.
  • yahmm - Hidden Markov Models for Python, implemented in Cython for speed and efficiency.
  • python-timbl - A Python extension module wrapping the full TiMBL C++ programming interface. Timbl is an elaborate k-Nearest Neighbours machine learning toolkit.
  • deap - Evolutionary algorithm framework.
  • pydeep - Deep Learning In Python.
  • mlxtend - A library consisting of useful tools for data science and machine learning tasks.
  • neon - Nervana's high-performance Python-based Deep Learning framework [DEEP LEARNING].
  • Optunity - A library dedicated to automated hyperparameter optimization with a simple, lightweight API to facilitate drop-in replacement of grid search.
  • Neural Networks and Deep Learning - Code samples for my book "Neural Networks and Deep Learning" [DEEP LEARNING].
  • Annoy - Approximate nearest neighbours implementation.
  • skflow - Simplified interface for TensorFlow, mimicking Scikit Learn.
  • TPOT - Tool that automatically creates and optimizes machine learning pipelines using genetic programming. Consider it your personal data science assistant, automating a tedious part of machine learning.
  • pgmpy A python library for working with Probabilistic Graphical Models.
  • DIGITS - The Deep Learning GPU Training System (DIGITS) is a web application for training deep learning models.
  • Orange - Open source data visualization and data analysis for novices and experts.
  • MXNet - Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Go, Javascript and more.
  • milk - Machine learning toolkit focused on supervised classification.
  • TFLearn - Deep learning library featuring a higher-level API for TensorFlow.
  • REP - an IPython-based environment for conducting data-driven research in a consistent and reproducible way. REP is not trying to substitute scikit-learn, but extends it and provides better user experience.
  • rgf_python - Python bindings for Regularized Greedy Forest (Tree) Library.
  • skbayes - Python package for Bayesian Machine Learning with scikit-learn API.
  • fuku-ml - Simple machine learning library, including Perceptron, Regression, Support Vector Machine, Decision Tree and more, it's easy to use and easy to learn for beginners.
  • Xcessiv - A web-based application for quick, scalable, and automated hyperparameter tuning and stacked ensembling.
  • PyTorch - Tensors and Dynamic neural networks in Python with strong GPU acceleration
  • ML-From-Scratch - Implementations of Machine Learning models from scratch in Python with a focus on transparency. Aims to showcase the nuts and bolts of ML in an accessible way.
  • Edward - A library for probabilistic modeling, inference, and criticism. Built on top of TensorFlow.
  • xRBM - A library for Restricted Boltzmann Machine (RBM) and its conditional variants in Tensorflow.
  • CatBoost - General purpose gradient boosting on decision trees library with categorical features support out of the box. It is easy to install, well documented and supports CPU and GPU (even multi-GPU) computation.
  • stacked_generalization - Implementation of machine learning stacking technic as handy library in Python.
  • modAL - A modular active learning framework for Python, built on top of scikit-learn.
  • Cogitare: A Modern, Fast, and Modular Deep Learning and Machine Learning framework for Python.
  • Parris - Parris, the automated infrastructure setup tool for machine learning algorithms.
  • neonrvm - neonrvm is an open source machine learning library based on RVM technique. It's written in C programming language and comes with Python programming language bindings.
  • Turi Create - Machine learning from Apple. Turi Create simplifies the development of custom machine learning models. You don't have to be a machine learning expert to add recommendations, object detection, image classification, image similarity or activity classification to your app.

Misc Scripts / iPython Notebooks / Codebases

Neural Networks

  • NeuralTalk - NeuralTalk is a Python+numpy project for learning Multimodal Recurrent Neural Networks that describe images with sentences.
  • Neuron - Neuron is simple class for time series predictions. It's utilize LNU (Linear Neural Unit), QNU (Quadratic Neural Unit), RBF (Radial Basis Function), MLP (Multi Layer Perceptron), MLP-ELM (Multi Layer Perceptron - Extreme Learning Machine) neural networks learned with Gradient descent or LeLevenberg–Marquardt algorithm.
  • Data Driven Code - Very simple implementation of neural networks for dummies in python without using any libraries, with detailed comments.

Kaggle Competition Source Code

Reinforcement Learning

  • DeepMind Lab - DeepMind Lab is a 3D learning environment based on id Software's Quake III Arena via ioquake3 and other open source software. Its primary purpose is to act as a testbed for research in artificial intelligence, especially deep reinforcement learning.
  • Gym - OpenAI Gym is a toolkit for developing and comparing reinforcement learning algorithms.
  • Serpent.AI - Serpent.AI is a game agent framework that allows you to turn any video game you own into a sandbox to develop AI and machine learning experiments. For both researchers and hobbyists.
  • Universe - Universe is a software platform for measuring and training an AI's general intelligence across the world's supply of games, websites and other applications.
  • ViZDoom - ViZDoom allows developing AI bots that play Doom using only the visual information (the screen buffer). It is primarily intended for research in machine visual learning, and deep reinforcement learning, in particular.