-
Notifications
You must be signed in to change notification settings - Fork 3
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Release v0.1.0: RandomNameGenerator implemented (#16)
Create project structure (#1): * Addded base dependencies * Create base struct of project and CI/CD config * Added config for linters and module description * Added setup config -------------------------------------------------------------------------------------------------------------------- Added datafile for random generator (#2) Implement interfaces and base functionality RandomNameGenerator (#3): Implement the base functionality of the RandomNameGenerator class and provide interfaces for BaseGenerator. * Provide base interfaces * Implement base functionality for RandomNameGenerator * Provide base constants * Code refactor -------------------------------------------------------------------------------------------------------------------- added RandomNameGenerator tests and errors (#4): * added RandomNameGenerator tests and errors * Update travis config * Added constants * Added ValidateDataError * requirements update * code refactor -------------------------------------------------------------------------------------------------------------------- Include `nounlist.txt` to the package data (#5): * include nounlist.txt to the package data * Added package installation testing * Added version to module (#6) * Added `__version__` and `__version_info__` to the top module eos_name_generator. -------------------------------------------------------------------------------------------------------------------- Added Setters and Getters in RandomNameGenerator (#7): * Update docstrings in RandomNameGenerator * Added `seed_data_path` setter and getter * Refactor constants names * Added `numbers_probabilities` Setter and Getter * RandomNameGenerator tests changes * Test changes RandomNameGenerator * Fixed Setters RandomNameGenerator * Docstring changes * Interface changes for `generate_list` method * Fixed `test_generete_list` -------------------------------------------------------------------------------------------------------------------- Added methods to calculate probabilities words (#8): Added methods to calculate probabilities of the length base word and the probability that additional words will be alphabetical. Since we have a name length of 12 characters, an additional word with the desired length may not exist in the dictionary; in this case, an additional words will be a set of numbers. * Added `get_probabilities_len_base_word` method * Added `get_probability_alphabet_additional_word` method -------------------------------------------------------------------------------------------------------------------- Implementation `generate` and `generate_list` methods in RandomNameGenerator (#9): Added a method for generating a name based on the base word, the probability of the length of the base word depends on their frequency. An additional word can consist of letters or numbers, the probability of getting numbers sets `numbers_probabilities` variable and existence additional alphabetical word in the base dictionary. * Added implementation of the `generate` method * Added implementation of the `generate_list` method * Added `get_random_name` method * Update test cases * Fixed MANIFEST.in include -------------------------------------------------------------------------------------------------------------------- Remove repeat from `nounlist.txt` (#10) -------------------------------------------------------------------------------------------------------------------- Fixed `random.randint` range (#12) -------------------------------------------------------------------------------------------------------------------- Random Generator Documentation (#13) -------------------------------------------------------------------------------------------------------------------- Added `random_provider` integration to `RandomNameGenerator` (#11): * Implement `random_provider` to `RandomNameGenerator` Added the ability to integrate your own `random_provider` which will be faster than the build-in (for example `random`, or `numpy`). Two providers are implemented for selection: * numpy.random * FastRandomChoice `FastRandomChoice` - is a class extend build-in `random` python package. Its provide `choice` method with probabilities selection. `FastRandomChoice.choice()` is 5 times faster than `numpy.random.choice`. Data was checked on a `5_000_000` sample. Added: * Added the ability to integrate your own random_provider to RandomNameGenerator * FastRandomChoice class implement * Tests -------------------------------------------------------------------------------------------------------------------- * Create `data_provider` for `RandomNameGenerator` (#14) * Create `data_provider` for `RandomNameGenerator` Moving code related to reading data to a separate class `DataReader` and create `random_provider` variable in `RandomNameGenerator`. This was done to comply with the `SOLID` programming principles. Implement: * Added DataReader class * Added `random_provider` variable with `setters` and `getter` at `RandomNameGenerator` * Update .travis config * `data_reader` tests -------------------------------------------------------------------------------------------------------------------- Changes in FastRandomChoice (#15) * Changes in FastRandomChoice Change native python loops to `numpy` `multidimensional_shifting`. `multidimensional_shifting` - is a method that gets the most probable an element from probabilities sequence. Algorithm speed - it's faster than `numpy.random.choice` less than 1.5x. Collision Resistance: `Samples`: 1_000_000 `numpy.random.choice`: 7790 collisions `FastRandomChoice`: 500 collisions `FastRandomChoice` more resistant to collisions than `numpy.random.choice` at ~5-10x times.
- Loading branch information
1 parent
dfe2b2a
commit 1f286b7
Showing
31 changed files
with
11,515 additions
and
21 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
coverage: | ||
precision: 2 | ||
round: down | ||
range: 70...100 | ||
|
||
status: | ||
patch: off |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -127,3 +127,6 @@ dmypy.json | |
|
||
# Pyre type checker | ||
.pyre/ | ||
|
||
# Pycharm project configuration | ||
.idea/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
package: | ||
name: eos-name-generator | ||
|
||
ci: | ||
name: travis | ||
|
||
branches: | ||
development: develop | ||
release: master |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
language: python | ||
|
||
sudo: required | ||
|
||
dist: xenial | ||
|
||
python: | ||
- "3.6" | ||
- "3.7" | ||
|
||
install: | ||
- pip install -r requirements.txt | ||
- pip install -r requirements-dev.txt | ||
|
||
script: | ||
- radon cc eos_name_generator -nb --total-average | ||
- cat requirements.txt requirements-dev.txt | safety check --stdin | ||
- isort -rc eos_name_generator tests --diff | ||
- flake8 eos_name_generator && flake8 tests | ||
- coverage run -m pytest -vv tests | ||
- python3 setup.py sdist && pip3 install dist/*.tar.gz | ||
|
||
after_success: | ||
- coverage report -m && coverage xml | ||
- bash <(curl -s https://codecov.io/bash) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
include *.md | ||
include LICENSE | ||
include requirements.txt | ||
include requirements-dev.txt | ||
include eos_name_generator/random_generator/seed_data/nounlist.txt | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,37 +1,175 @@ | ||
## Welcome to GitHub Pages | ||
## EOS Name Generator | ||
|
||
You can use the [editor on GitHub](https://github.com/Alladin9393/eos-name-generator/edit/master/README.md) to maintain and preview the content for your website in Markdown files. | ||
[](https://travis-ci.com/Alladin9393/eos-name-generator) | ||
[](https://www.codefactor.io/repository/github/alladin9393/eos-name-generator) | ||
[](https://codecov.io/gh/Alladin9393/eos-name-generator) | ||
|
||
Whenever you commit to this repository, GitHub Pages will run [Jekyll](https://jekyllrb.com/) to rebuild the pages in your site, from the content in your Markdown files. | ||
`EOS Name Generator` - is a python package for generating names according to [EOS name rules](#eos-name-rules). | ||
|
||
### Markdown | ||
This includes 3 ways to generate a name: | ||
|
||
Markdown is a lightweight and easy-to-use syntax for styling your writing. It includes conventions for | ||
* Random Generator | ||
* Recurrent Neural Network Generator | ||
* Markov Chain Generator | ||
|
||
* [Getting started](#getting-started) | ||
* [Requirements](#getting-started-requirements) | ||
* [Ubuntu 16.04 & 18.04](#ubuntu-1604--1804) | ||
* [MacOS](#macos) | ||
* [Installation](#installation) | ||
* [Algorithm description](#algorithm-description) | ||
* [Random Generator](#random-generator) | ||
* [Recurrent Neural Network Generator](#recurrent-neural-network-generator) | ||
* [Markov Chain Generator](#markov-chain-generator) | ||
* [EOS Name Rules](#eos-name-rules) | ||
* [Usage](#usage) | ||
* [Service](#service) | ||
* [Random Generator](#random-generator-usage) | ||
* [Recurrent Neural Network Generator](#recurrent-neural-network-generator-usage) | ||
* [Markov Chain Generator](#markov-chain-generator-usage) | ||
* [Development](#development) | ||
* [Production](#production) | ||
* [Contributing](#contributing) | ||
* [Request pull request's review](#request-pull-requests-review) | ||
|
||
```markdown | ||
Syntax highlighted code block | ||
## Getting started | ||
|
||
# Header 1 | ||
## Header 2 | ||
### Header 3 | ||
<h3 id="getting-started-requirements">Requirements</h4> | ||
|
||
- Bulleted | ||
- List | ||
#### Ubuntu 16.04 & 18.04 | ||
|
||
1. Numbered | ||
2. List | ||
If you have 16.04 version, install system requirements with the following terminal commands: | ||
|
||
**Bold** and _Italic_ and `Code` text | ||
```bash | ||
$ sudo apt update && sudo apt install -y software-properties-common build-essential | ||
``` | ||
|
||
#### MacOS | ||
|
||
Install Python 3.7 (also, we support 3.6): | ||
```bash | ||
$ brew install python3 | ||
``` | ||
|
||
## Installation | ||
|
||
Install the package from the [PyPi](https://pypi.org/project/eos-name-generator) through pip: | ||
|
||
[Link](url) and  | ||
```bash | ||
$ pip3 install eos-name-generator | ||
``` | ||
|
||
For more details see [GitHub Flavored Markdown](https://guides.github.com/features/mastering-markdown/). | ||
## Algorithm description | ||
|
||
#### Random Generator | ||
|
||
The algorithm selects a base word from a pre-prepared dictionary, after which an additional word of the appropriate | ||
length is selected `ADDITIONAL_WORD_LEN = EOS_NAME_LEN - BASE_WORD_LEN` if there is no such word, then the additional | ||
word is random numbers. | ||
|
||
The probability that an additional word will be numbers can be set by the parameter `numbers_probabilities`. | ||
|
||
#### Recurrent Neural Network Generator | ||
|
||
#### Markov Chain Generator | ||
|
||
#### EOS Name Rules | ||
|
||
* Can only contain the characters `.abcdefghijklmnopqrstuvwxyz12345`. `a-z` (lowercase), `1-5` and `.` (period) | ||
* Must start with a letter | ||
* Must be `12` characters | ||
|
||
## Usage | ||
|
||
#### Service | ||
|
||
Get the version of the package — `eos-name-generator --version`: | ||
|
||
```bash | ||
$ eos-name-generator --version | ||
``` | ||
|
||
#### Random Generator | ||
<a name="random-generator-usage"></a> | ||
|
||
Generate random name: | ||
|
||
```python | ||
from eos_name_generator import RandomNameGenerator | ||
|
||
if __name__ == '__main__': | ||
generator = RandomNameGenerator() | ||
name = generator.generate() | ||
print(name) | ||
``` | ||
|
||
Generate list of random names: | ||
|
||
```python | ||
from eos_name_generator import RandomNameGenerator | ||
|
||
if __name__ == '__main__': | ||
generator = RandomNameGenerator() | ||
names = generator.generate_list(num=1000) | ||
|
||
for name in names: | ||
print(name) | ||
``` | ||
|
||
#### Recurrent Neural Network Generator | ||
<a name="recurrent-neural-network-generator-usage"></a> | ||
|
||
#### Markov Chain Generator | ||
<a name="markov-chain-generator-usage"></a> | ||
|
||
## Development | ||
|
||
Clone the project and move to project folder: | ||
|
||
```bash | ||
$ git clone https://github.com/Alladin9393/eos-name-generator.git && cd eos-name-generator | ||
``` | ||
|
||
Create virtualenv and install requirements: | ||
|
||
```bash | ||
$ virtualenv venv -p python3 && source venv/bin/activate | ||
$ pip3 install -r requirements.txt -r requirements-dev.txt | ||
``` | ||
|
||
To run tests use: | ||
|
||
```bash | ||
$ coverage run -m pytest -vv tests | ||
``` | ||
|
||
When you have developed new functionality, check it with the following command. This command creates the Python | ||
package from source code instead of installing it from the PyPi: | ||
|
||
```bash | ||
$ pip3 uninstall -y eos_name_generator && rm -rf dist/ eos_name_generator.egg-info && \ | ||
python3 setup.py sdist && pip3 install dist/*.tar.gz | ||
``` | ||
## Production | ||
|
||
To build the package and upload it to [PypI](https://pypi.org/) to be accessible through | ||
[pip](https://github.com/pypa/pip), use the following commands. [Twine](https://twine.readthedocs.io/en/latest/) | ||
requires the username and password of the account package is going to be uploaded to. | ||
|
||
```bash | ||
$ python3 setup.py sdist | ||
$ twine upload dist/* | ||
username: alladin9393 | ||
password: ****** | ||
``` | ||
|
||
### Jekyll Themes | ||
## Contributing | ||
|
||
Your Pages site will use the layout and styles from the Jekyll theme you have selected in your [repository settings](https://github.com/Alladin9393/eos-name-generator/settings). The name of this theme is saved in the Jekyll `_config.yml` configuration file. | ||
#### Request pull request's review | ||
|
||
### Support or Contact | ||
If you want to your pull request to be review, ensure you: | ||
|
||
Having trouble with Pages? Check out our [documentation](https://help.github.com/categories/github-pages-basics/) or [contact support](https://github.com/contact) and we’ll help you sort it out. | ||
If you want to your pull request to be review, ensure you: | ||
1. [Branch isn't out-of-date with the base branch](https://habrastorage.org/webt/ux/gi/wm/uxgiwmnft08fubvjfd6d-8pw2wq.png). | ||
2. [Have written the description of the pull request and have added at least 2 reviewers](https://camo.githubusercontent.com/55c309334a8b61a4848a6ef25f9b0fb3751ae5e9/68747470733a2f2f686162726173746f726167652e6f72672f776562742f74312f70792f63752f7431707963753162786a736c796f6a6c707935306d7862357969652e706e67). | ||
3. [Continuous integration has been passed](https://habrastorage.org/webt/oz/fl/-n/ozfl-nl-jynrh7ofz8yuz9_gapy.png). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
"""Top-level module for `EOS` name generator. | ||
This module generate random name which suits for `eosio blockchain` name | ||
conversation. | ||
The module provides 3 methods to generate a name: | ||
- random name generation based in a ready-made data (word dictionary) | ||
- markov chain text generation | ||
- recurrent neural network text generation | ||
""" | ||
from eos_name_generator.random_generator.generator import RandomNameGenerator | ||
|
||
__version__ = "0.1.0" | ||
__version_info__ = tuple( | ||
int(i) for i in __version__.split(".") if i.isdigit() | ||
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
""" | ||
Provide constants for BaseGenerator interface. | ||
""" | ||
from os.path import dirname | ||
|
||
from eos_name_generator.utils import FastRandomChoice | ||
|
||
EOS_NAME_LENGTH = 12 | ||
SEED_DATA_PATH = dirname(__file__) + '/' + 'random_generator/seed_data/nounlist.txt' | ||
NUMBERS_PROBABILITIES = 0.1 | ||
RANDOM_PROVIDER_INSTANCE = FastRandomChoice() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
""" | ||
Provide errors for BaseGenerator interface. | ||
""" | ||
|
||
|
||
class ValidationDataError(Exception): | ||
""" | ||
Name generation data contains invalid characters or does not match the name length error. | ||
""" | ||
|
||
def __init__(self, message): | ||
self.message = message |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
""" | ||
Provide implementation of the BaseGenerator interfaces. | ||
""" | ||
from abc import ( | ||
ABC, | ||
abstractmethod, | ||
) | ||
|
||
|
||
class BaseGeneratorInterface(ABC): | ||
""" | ||
Implements BaseGenerator interface. | ||
""" | ||
|
||
@abstractmethod | ||
def generate(self) -> str: | ||
""" | ||
Generate `EOS` name. | ||
:return: `EOS` name | ||
""" | ||
|
||
@abstractmethod | ||
def generate_list(self, num: int) -> list: | ||
""" | ||
Generate list of `EOS` names. | ||
:param num: number of generated names in list. | ||
:return: `EOS` name | ||
""" |
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
""" | ||
Provide an implementation of the MarkovChainGenerator interface. | ||
""" |
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,48 @@ | ||
""" | ||
Provide implementation of the DataReader. | ||
""" | ||
from collections import defaultdict | ||
|
||
from eos_name_generator.constants import EOS_NAME_LENGTH | ||
from eos_name_generator.errors import ValidationDataError | ||
|
||
|
||
class DataReader: | ||
""" | ||
Implementation of the DataReader. | ||
""" | ||
|
||
def __init__(self, data_path): | ||
self.data_path = data_path | ||
|
||
def get_dictionary_by_word_len(self) -> defaultdict: | ||
""" | ||
Read data from `seed_data_path`. | ||
Read data from `seed_data_path` and transform it into `dictionary` object | ||
where the key is the word length. | ||
""" | ||
with open(self.data_path) as f: | ||
data = f.read().splitlines() | ||
|
||
self.__validate_data(data) | ||
data_dictionary_by_word_len = defaultdict(list) | ||
for word in data: | ||
word_len = len(word) | ||
data_dictionary_by_word_len[word_len].append(word) | ||
|
||
return data_dictionary_by_word_len | ||
|
||
@staticmethod | ||
def __validate_data(data: list): | ||
""" | ||
Seed data validation to generate the correct `eos` name. | ||
:param data: data to be validated | ||
""" | ||
for word in data: | ||
is_valid_word_len = (len(word) <= EOS_NAME_LENGTH) | ||
is_valid_word = word.isalpha() and word.islower() and is_valid_word_len | ||
|
||
if not is_valid_word: | ||
raise ValidationDataError("Data contains invalid characters or does not match the name length error") |
Oops, something went wrong.