Skip to content

Commit

Permalink
Release v0.1.0: RandomNameGenerator implemented (#16)
Browse files Browse the repository at this point in the history
Create project structure (#1):

* Addded base dependencies
* Create base struct of project and CI/CD config
* Added config for linters and module description
* Added setup config
--------------------------------------------------------------------------------------------------------------------
Added datafile for random generator (#2)

Implement interfaces and base functionality RandomNameGenerator (#3):

Implement the base functionality of the RandomNameGenerator class and
provide interfaces for BaseGenerator.

* Provide base interfaces
* Implement base functionality for RandomNameGenerator
* Provide base constants
* Code refactor
--------------------------------------------------------------------------------------------------------------------
added RandomNameGenerator tests and errors (#4):

* added RandomNameGenerator tests and errors
* Update travis config
* Added constants
* Added ValidateDataError
* requirements update
* code refactor
--------------------------------------------------------------------------------------------------------------------
Include `nounlist.txt` to the package data  (#5):

* include nounlist.txt to the package data
* Added package installation testing
* Added version to module (#6)
* Added `__version__` and `__version_info__` to the top module eos_name_generator.
--------------------------------------------------------------------------------------------------------------------
Added Setters and Getters in RandomNameGenerator (#7):

* Update docstrings in RandomNameGenerator
* Added `seed_data_path` setter and getter
* Refactor constants names
* Added `numbers_probabilities` Setter and Getter
* RandomNameGenerator tests changes
* Test changes RandomNameGenerator
* Fixed Setters RandomNameGenerator
* Docstring changes
* Interface changes for `generate_list` method
* Fixed `test_generete_list`
--------------------------------------------------------------------------------------------------------------------
Added methods to calculate probabilities words (#8):

Added methods to calculate probabilities of the length base word and
the probability that additional words will be alphabetical.

Since we have a name length of 12 characters, an additional word with
the desired length may not exist in the dictionary; in this case, an
additional words will be a set of numbers.

* Added `get_probabilities_len_base_word` method
* Added `get_probability_alphabet_additional_word` method
--------------------------------------------------------------------------------------------------------------------
Implementation `generate` and `generate_list` methods in RandomNameGenerator (#9):

Added a method for generating a name based on the base word, the
probability of the length of the base word depends on their frequency.

An additional word can consist of letters or numbers, the probability of
getting numbers sets `numbers_probabilities` variable and existence
additional alphabetical word in the base dictionary.

* Added implementation of the `generate` method
* Added implementation of the `generate_list` method
* Added `get_random_name` method
* Update test cases
* Fixed MANIFEST.in include
--------------------------------------------------------------------------------------------------------------------
Remove repeat from `nounlist.txt` (#10)
--------------------------------------------------------------------------------------------------------------------
Fixed `random.randint` range (#12)
--------------------------------------------------------------------------------------------------------------------
Random Generator Documentation (#13)
--------------------------------------------------------------------------------------------------------------------
Added `random_provider` integration to `RandomNameGenerator` (#11):

* Implement `random_provider` to `RandomNameGenerator`

Added the ability to integrate your own `random_provider` which
will be faster than the build-in (for example `random`, or `numpy`).

Two providers are implemented for selection:
* numpy.random
* FastRandomChoice

`FastRandomChoice` - is a class extend build-in `random` python
package. Its provide `choice` method with probabilities selection.

`FastRandomChoice.choice()` is 5 times faster than `numpy.random.choice`.
Data was checked on a `5_000_000` sample.

Added: 
* Added the ability to integrate your own random_provider to RandomNameGenerator
* FastRandomChoice class implement
* Tests
--------------------------------------------------------------------------------------------------------------------
* Create `data_provider` for `RandomNameGenerator` (#14)

* Create `data_provider` for `RandomNameGenerator`

Moving code related to reading data to a separate class `DataReader`
and create `random_provider` variable in `RandomNameGenerator`.

This was done to comply with the `SOLID` programming principles.

Implement:
* Added DataReader class
* Added `random_provider` variable with `setters` and `getter` at
`RandomNameGenerator`
* Update .travis config
* `data_reader` tests
--------------------------------------------------------------------------------------------------------------------
Changes in FastRandomChoice (#15)

* Changes in FastRandomChoice

Change native python loops to `numpy` `multidimensional_shifting`.
`multidimensional_shifting` - is a method that gets the most probable
an element from probabilities sequence.

Algorithm speed - it's faster than `numpy.random.choice` less than 1.5x.

Collision Resistance:
`Samples`: 1_000_000
`numpy.random.choice`: 7790 collisions
`FastRandomChoice`: 500 collisions

`FastRandomChoice` more resistant to collisions than
`numpy.random.choice` at ~5-10x times.
  • Loading branch information
Alladin9393 authored Jun 4, 2020
1 parent dfe2b2a commit 1f286b7
Show file tree
Hide file tree
Showing 31 changed files with 11,515 additions and 21 deletions.
7 changes: 7 additions & 0 deletions .codecov.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
coverage:
precision: 2
round: down
range: 70...100

status:
patch: off
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -127,3 +127,6 @@ dmypy.json

# Pyre type checker
.pyre/

# Pycharm project configuration
.idea/
9 changes: 9 additions & 0 deletions .pypi-version.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
package:
name: eos-name-generator

ci:
name: travis

branches:
development: develop
release: master
25 changes: 25 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
language: python

sudo: required

dist: xenial

python:
- "3.6"
- "3.7"

install:
- pip install -r requirements.txt
- pip install -r requirements-dev.txt

script:
- radon cc eos_name_generator -nb --total-average
- cat requirements.txt requirements-dev.txt | safety check --stdin
- isort -rc eos_name_generator tests --diff
- flake8 eos_name_generator && flake8 tests
- coverage run -m pytest -vv tests
- python3 setup.py sdist && pip3 install dist/*.tar.gz

after_success:
- coverage report -m && coverage xml
- bash <(curl -s https://codecov.io/bash)
6 changes: 6 additions & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
include *.md
include LICENSE
include requirements.txt
include requirements-dev.txt
include eos_name_generator/random_generator/seed_data/nounlist.txt

180 changes: 159 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,37 +1,175 @@
## Welcome to GitHub Pages
## EOS Name Generator

You can use the [editor on GitHub](https://github.com/Alladin9393/eos-name-generator/edit/master/README.md) to maintain and preview the content for your website in Markdown files.
[![Build Status](https://travis-ci.com/Alladin9393/eos-name-generator.svg?branch=develop)](https://travis-ci.com/Alladin9393/eos-name-generator)
[![CodeFactor](https://www.codefactor.io/repository/github/alladin9393/eos-name-generator/badge)](https://www.codefactor.io/repository/github/alladin9393/eos-name-generator)
[![codecov](https://codecov.io/gh/Alladin9393/eos-name-generator/branch/develop/graph/badge.svg)](https://codecov.io/gh/Alladin9393/eos-name-generator)

Whenever you commit to this repository, GitHub Pages will run [Jekyll](https://jekyllrb.com/) to rebuild the pages in your site, from the content in your Markdown files.
`EOS Name Generator` - is a python package for generating names according to [EOS name rules](#eos-name-rules).

### Markdown
This includes 3 ways to generate a name:

Markdown is a lightweight and easy-to-use syntax for styling your writing. It includes conventions for
* Random Generator
* Recurrent Neural Network Generator
* Markov Chain Generator

* [Getting started](#getting-started)
* [Requirements](#getting-started-requirements)
* [Ubuntu 16.04 & 18.04](#ubuntu-1604--1804)
* [MacOS](#macos)
* [Installation](#installation)
* [Algorithm description](#algorithm-description)
* [Random Generator](#random-generator)
* [Recurrent Neural Network Generator](#recurrent-neural-network-generator)
* [Markov Chain Generator](#markov-chain-generator)
* [EOS Name Rules](#eos-name-rules)
* [Usage](#usage)
* [Service](#service)
* [Random Generator](#random-generator-usage)
* [Recurrent Neural Network Generator](#recurrent-neural-network-generator-usage)
* [Markov Chain Generator](#markov-chain-generator-usage)
* [Development](#development)
* [Production](#production)
* [Contributing](#contributing)
* [Request pull request's review](#request-pull-requests-review)

```markdown
Syntax highlighted code block
## Getting started

# Header 1
## Header 2
### Header 3
<h3 id="getting-started-requirements">Requirements</h4>

- Bulleted
- List
#### Ubuntu 16.04 & 18.04

1. Numbered
2. List
If you have 16.04 version, install system requirements with the following terminal commands:

**Bold** and _Italic_ and `Code` text
```bash
$ sudo apt update && sudo apt install -y software-properties-common build-essential
```

#### MacOS

Install Python 3.7 (also, we support 3.6):
```bash
$ brew install python3
```

## Installation

Install the package from the [PyPi](https://pypi.org/project/eos-name-generator) through pip:

[Link](url) and ![Image](src)
```bash
$ pip3 install eos-name-generator
```

For more details see [GitHub Flavored Markdown](https://guides.github.com/features/mastering-markdown/).
## Algorithm description

#### Random Generator

The algorithm selects a base word from a pre-prepared dictionary, after which an additional word of the appropriate
length is selected `ADDITIONAL_WORD_LEN = EOS_NAME_LEN - BASE_WORD_LEN` if there is no such word, then the additional
word is random numbers.

The probability that an additional word will be numbers can be set by the parameter `numbers_probabilities`.

#### Recurrent Neural Network Generator

#### Markov Chain Generator

#### EOS Name Rules

* Can only contain the characters `.abcdefghijklmnopqrstuvwxyz12345`. `a-z` (lowercase), `1-5` and `.` (period)
* Must start with a letter
* Must be `12` characters

## Usage

#### Service

Get the version of the package — `eos-name-generator --version`:

```bash
$ eos-name-generator --version
```

#### Random Generator
<a name="random-generator-usage"></a>

Generate random name:

```python
from eos_name_generator import RandomNameGenerator

if __name__ == '__main__':
generator = RandomNameGenerator()
name = generator.generate()
print(name)
```

Generate list of random names:

```python
from eos_name_generator import RandomNameGenerator

if __name__ == '__main__':
generator = RandomNameGenerator()
names = generator.generate_list(num=1000)

for name in names:
print(name)
```

#### Recurrent Neural Network Generator
<a name="recurrent-neural-network-generator-usage"></a>

#### Markov Chain Generator
<a name="markov-chain-generator-usage"></a>

## Development

Clone the project and move to project folder:

```bash
$ git clone https://github.com/Alladin9393/eos-name-generator.git && cd eos-name-generator
```

Create virtualenv and install requirements:

```bash
$ virtualenv venv -p python3 && source venv/bin/activate
$ pip3 install -r requirements.txt -r requirements-dev.txt
```

To run tests use:

```bash
$ coverage run -m pytest -vv tests
```

When you have developed new functionality, check it with the following command. This command creates the Python
package from source code instead of installing it from the PyPi:

```bash
$ pip3 uninstall -y eos_name_generator && rm -rf dist/ eos_name_generator.egg-info && \
python3 setup.py sdist && pip3 install dist/*.tar.gz
```
## Production

To build the package and upload it to [PypI](https://pypi.org/) to be accessible through
[pip](https://github.com/pypa/pip), use the following commands. [Twine](https://twine.readthedocs.io/en/latest/)
requires the username and password of the account package is going to be uploaded to.

```bash
$ python3 setup.py sdist
$ twine upload dist/*
username: alladin9393
password: ******
```

### Jekyll Themes
## Contributing

Your Pages site will use the layout and styles from the Jekyll theme you have selected in your [repository settings](https://github.com/Alladin9393/eos-name-generator/settings). The name of this theme is saved in the Jekyll `_config.yml` configuration file.
#### Request pull request's review

### Support or Contact
If you want to your pull request to be review, ensure you:

Having trouble with Pages? Check out our [documentation](https://help.github.com/categories/github-pages-basics/) or [contact support](https://github.com/contact) and we’ll help you sort it out.
If you want to your pull request to be review, ensure you:
1. [Branch isn't out-of-date with the base branch](https://habrastorage.org/webt/ux/gi/wm/uxgiwmnft08fubvjfd6d-8pw2wq.png).
2. [Have written the description of the pull request and have added at least 2 reviewers](https://camo.githubusercontent.com/55c309334a8b61a4848a6ef25f9b0fb3751ae5e9/68747470733a2f2f686162726173746f726167652e6f72672f776562742f74312f70792f63752f7431707963753162786a736c796f6a6c707935306d7862357969652e706e67).
3. [Continuous integration has been passed](https://habrastorage.org/webt/oz/fl/-n/ozfl-nl-jynrh7ofz8yuz9_gapy.png).
16 changes: 16 additions & 0 deletions eos_name_generator/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
"""Top-level module for `EOS` name generator.
This module generate random name which suits for `eosio blockchain` name
conversation.
The module provides 3 methods to generate a name:
- random name generation based in a ready-made data (word dictionary)
- markov chain text generation
- recurrent neural network text generation
"""
from eos_name_generator.random_generator.generator import RandomNameGenerator

__version__ = "0.1.0"
__version_info__ = tuple(
int(i) for i in __version__.split(".") if i.isdigit()
)
11 changes: 11 additions & 0 deletions eos_name_generator/constants.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
"""
Provide constants for BaseGenerator interface.
"""
from os.path import dirname

from eos_name_generator.utils import FastRandomChoice

EOS_NAME_LENGTH = 12
SEED_DATA_PATH = dirname(__file__) + '/' + 'random_generator/seed_data/nounlist.txt'
NUMBERS_PROBABILITIES = 0.1
RANDOM_PROVIDER_INSTANCE = FastRandomChoice()
12 changes: 12 additions & 0 deletions eos_name_generator/errors.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
"""
Provide errors for BaseGenerator interface.
"""


class ValidationDataError(Exception):
"""
Name generation data contains invalid characters or does not match the name length error.
"""

def __init__(self, message):
self.message = message
30 changes: 30 additions & 0 deletions eos_name_generator/interfaces.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
"""
Provide implementation of the BaseGenerator interfaces.
"""
from abc import (
ABC,
abstractmethod,
)


class BaseGeneratorInterface(ABC):
"""
Implements BaseGenerator interface.
"""

@abstractmethod
def generate(self) -> str:
"""
Generate `EOS` name.
:return: `EOS` name
"""

@abstractmethod
def generate_list(self, num: int) -> list:
"""
Generate list of `EOS` names.
:param num: number of generated names in list.
:return: `EOS` name
"""
Empty file.
3 changes: 3 additions & 0 deletions eos_name_generator/markov_chain_generator/generator.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
"""
Provide an implementation of the MarkovChainGenerator interface.
"""
Empty file.
48 changes: 48 additions & 0 deletions eos_name_generator/random_generator/data_reader.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
"""
Provide implementation of the DataReader.
"""
from collections import defaultdict

from eos_name_generator.constants import EOS_NAME_LENGTH
from eos_name_generator.errors import ValidationDataError


class DataReader:
"""
Implementation of the DataReader.
"""

def __init__(self, data_path):
self.data_path = data_path

def get_dictionary_by_word_len(self) -> defaultdict:
"""
Read data from `seed_data_path`.
Read data from `seed_data_path` and transform it into `dictionary` object
where the key is the word length.
"""
with open(self.data_path) as f:
data = f.read().splitlines()

self.__validate_data(data)
data_dictionary_by_word_len = defaultdict(list)
for word in data:
word_len = len(word)
data_dictionary_by_word_len[word_len].append(word)

return data_dictionary_by_word_len

@staticmethod
def __validate_data(data: list):
"""
Seed data validation to generate the correct `eos` name.
:param data: data to be validated
"""
for word in data:
is_valid_word_len = (len(word) <= EOS_NAME_LENGTH)
is_valid_word = word.isalpha() and word.islower() and is_valid_word_len

if not is_valid_word:
raise ValidationDataError("Data contains invalid characters or does not match the name length error")
Loading

0 comments on commit 1f286b7

Please sign in to comment.