Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update requirements #127

Merged
merged 20 commits into from
Oct 23, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 58 additions & 0 deletions .github/workflows/test_and_deploy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
name: tests

on:
push:
branches:
- main
tags:
- "v*" # Push events to matching v*, i.e. v1.0, v20.15.10
pull_request:
branches:
- main
workflow_dispatch:

jobs:
test:
name: ${{ matrix.platform }} py${{ matrix.python-version }}
runs-on: ${{ matrix.platform }}
strategy:
matrix:
platform: [ubuntu-latest, windows-latest, macos-latest]
python-version: [3.7, 3.8, 3.9, '3.10']

steps:
- uses: actions/checkout@v3

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v3
with:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install setuptools tox tox-gh-actions
- name: Test with tox
run: tox
deploy:
needs: [test]
runs-on: ubuntu-latest
if: contains(github.ref, 'tags')
steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v3
with:
python-version: "3.x"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -U setuptools setuptools_scm wheel twine build
- name: Build and publish
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.TWINE_API_KEY }}
run: |
git tag
python -m build .
twine upload dist/*
26 changes: 0 additions & 26 deletions .travis.yml

This file was deleted.

20 changes: 14 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,23 +24,31 @@ OpenReview: [https://openreview.net/forum?id=IZfQYb4lHVq](https://openreview.net

## Installation
This implementation requires [Tensorflow](https://www.tensorflow.org/install/).
We have tested Noise2Void using Python 3.7 and tensorflow-gpu 2.4.1.
We have tested Noise2Void using Python 3.9 and TensorFlow 2.7 and 2.10.

Note: If you want to use TensorFlow 1.15 you have to install N2V v0.2.1. N2V v0.3.0 supports TensorFlow 2 only.
Note: If you want to use TensorFlow 1.15 you have to install N2V v0.2.1. N2V v0.3.* supports TensorFlow 2 only.

#### If you start from scratch...
We recommend using [miniconda](https://docs.conda.io/en/latest/miniconda.html).
If you do not yet have a strong opinion, just use it too!

After installing Miniconda, the following lines might are likely the easiest way to get Tensorflow and CuDNN installed on your machine (_Note:_ Macs are not supported, and if you sit on a Windows machine all this might also require some modifications.):
After installing Miniconda, create a conda environment:

```
conda create -n n2v python=3.8 cudatoolkit=11.0 cudnn=8.0 -c conda-forge
conda create -n 'n2v' python=3.9
conda activate n2v
pip install tensorflow==2.4
```

Once this is done (or you had tensorflow et al. installed already), you can install N2V with one of the following two options:
#### Install TensorFlow

The best way to install is to follow the [Tensorflow guidelines](https://www.tensorflow.org/install/pip).

Note that running the following commands in your environment will allow you to use the GPU without having to each
time run an `export` command (refer to the [Tensorflow step by step](https://www.tensorflow.org/install/pip#linux_1)):
```bash
mkdir -p $CONDA_PREFIX/etc/conda/activate.d
echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CONDA_PREFIX/lib/' > $CONDA_PREFIX/etc/conda/activate.d/env_vars.sh
```

#### Option 1: PIP (current stable release)
```
Expand Down
6 changes: 3 additions & 3 deletions n2v/internals/N2V_DataWrapper.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,12 +45,12 @@ def __init__(self, X, Y, batch_size, length, perc_pix=0.198, shape=(64, 64),

if self.dims == 2:
self.patch_sampler = self.__subpatch_sampling2D__
self.box_size = np.round(np.sqrt(100/perc_pix)).astype(np.int)
self.box_size = np.round(np.sqrt(100/perc_pix)).astype(np.int32)
self.get_stratified_coords = self.__get_stratified_coords2D__
self.rand_float = self.__rand_float_coords2D__(self.box_size)
elif self.dims == 3:
self.patch_sampler = self.__subpatch_sampling3D__
self.box_size = np.round(np.sqrt(100 / perc_pix)).astype(np.int)
self.box_size = np.round(np.sqrt(100 / perc_pix)).astype(np.int32)
self.get_stratified_coords = self.__get_stratified_coords3D__
self.rand_float = self.__rand_float_coords3D__(self.box_size)
else:
Expand Down Expand Up @@ -100,7 +100,7 @@ def apply_structN2Vmask(self, patch, coords, dims, mask):
each point in coords corresponds to the center of the mask.
then for point in the mask with value=1 we assign a random value
"""
coords = np.array(coords).astype(np.int)
coords = np.array(coords).astype(np.int32)
ndim = mask.ndim
center = np.array(mask.shape)//2
## leave the center value alone
Expand Down
8 changes: 4 additions & 4 deletions n2v/models/n2v_standard.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
tta_forward, tta_backward
from ..nets.unet import build_single_unet_per_channel, build_unet

from tifffile import imsave
from tifffile import imwrite
from csbdeep.utils.six import tempfile
import shutil

Expand Down Expand Up @@ -300,7 +300,7 @@ def prepare_for_training(self, optimizer=None, **kwargs):
"""
if optimizer is None:
from tensorflow.keras.optimizers import Adam
optimizer = Adam(lr=self.config.train_learning_rate)
optimizer = Adam(learning_rate=self.config.train_learning_rate)
self.callbacks = self.prepare_model(self.keras_model, optimizer, self.config.train_loss, **kwargs)

if self.basedir is not None:
Expand Down Expand Up @@ -486,8 +486,8 @@ def export_TF(self, name, description, authors, test_img, axes, patch_shape, fna

input_file = self.logdir / 'testinput.tif'
output_file = self.logdir / 'testoutput.tif'
imsave(input_file, test_img)
imsave(output_file, test_output)
imwrite(input_file, test_img)
imwrite(output_file, test_output)

with ZipFile(fname, 'a') as myzip:
myzip.write(yml_file, arcname=os.path.basename(yml_file))
Expand Down
7 changes: 7 additions & 0 deletions n2v/nets/blurpool.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,3 +60,10 @@ def compute_output_shape(self, input_shape):
int(np.ceil(input_shape[2] / 2)),
input_shape[3],
)

def get_config(self):
config = super().get_config()
config.update({
"pool": self.pool
})
return config
5 changes: 3 additions & 2 deletions n2v/nets/unet.py
Original file line number Diff line number Diff line change
Expand Up @@ -113,8 +113,9 @@ def build_unet(input_shape,

final = conv(num_channels, (1,)*n_dim, activation='linear')(unet)
if residual:
if not (num_channels == 1 if backend_channels_last() else num_channels
== 1):
if not (num_channels == 1):
#if not (num_channels == 1 if backend_channels_last() else num_channels
# == 1):
raise ValueError("number of input and output channels must be the same for a residual net.")
final = Add()([final, input])
final = Activation(activation=last_activation)(final)
Expand Down
4 changes: 2 additions & 2 deletions n2v/utils/n2v_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -162,11 +162,11 @@ def identity(patch, coords, dims, structN2Vmask=None):
def manipulate_val_data(X_val, Y_val, perc_pix=0.198, shape=(64, 64), value_manipulation=pm_uniform_withCP(5)):
dims = len(shape)
if dims == 2:
box_size = np.round(np.sqrt(100 / perc_pix)).astype(np.int)
box_size = np.round(np.sqrt(100 / perc_pix)).astype(np.int32)
get_stratified_coords = dw.__get_stratified_coords2D__
rand_float = dw.__rand_float_coords2D__(box_size)
elif dims == 3:
box_size = np.round(np.sqrt(100 / perc_pix)).astype(np.int)
box_size = np.round(np.sqrt(100 / perc_pix)).astype(np.int32)
get_stratified_coords = dw.__get_stratified_coords3D__
rand_float = dw.__rand_float_coords3D__(box_size)

Expand Down
3 changes: 3 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[build-system]
requires = ["setuptools", "wheel"]
build-backend = "setuptools.build_meta"
26 changes: 14 additions & 12 deletions setup.py
100755 → 100644
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,12 @@

_dir = path.abspath(path.dirname(__file__))

with open(path.join(_dir,'n2v','version.py')) as f:
with open(path.join(_dir, 'n2v', 'version.py')) as f:
exec(f.read())

with open(path.join(_dir,'README.md')) as f:
with open(path.join(_dir, 'README.md')) as f:
long_description = f.read()


setup(name='n2v',
version=__version__,
description='Noise2Void allows the training of a denoising CNN from individual noisy images. This implementation'
Expand All @@ -33,27 +32,30 @@
'Topic :: Scientific/Engineering',
'License :: OSI Approved :: BSD License',

'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
'Programming Language :: Python :: 3.10',
],


scripts=[
'scripts/trainN2V.py',
'scripts/predictN2V.py'
],


install_requires=[
"numpy",
"scipy",
"matplotlib",
"six",
"keras>=2.1.1,<2.4.0",
"tifffile>=2020.5.11",
"tifffile",
"imagecodecs>=2020.2.18",
"tqdm",
"backports.tempfile;python_version<'3.4'",
"csbdeep>=0.6.0,<0.7.0",
"csbdeep>=0.7.2,<0.8.0",
"Pillow",
"ruamel.yaml>=0.16.10"
]
],

extras_require={
"testing": ["pytest"]
}
)
4 changes: 1 addition & 3 deletions tests/test_Noise2VoidDataGenerator.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@


def test_generate_patches_2D():

if not os.path.isdir('data'):
os.mkdir('data')
zip_path = "data/RGB.zip"
Expand All @@ -25,8 +24,8 @@ def test_generate_patches_2D():
patches = datagen.generate_patches_from_list(imgs, shape=(110, 280))
assert len(patches) == 100

def test_generate_patches_3D():

def test_generate_patches_3D():
if not os.path.isdir('data'):
os.mkdir('data')
zip_path = 'data/flywing-data.zip'
Expand All @@ -44,4 +43,3 @@ def test_generate_patches_3D():
assert len(patches) == 1
patches = datagen.generate_patches_from_list(imgs[:1], shape=(5, 52, 174))
assert len(patches) == 210

2 changes: 1 addition & 1 deletion tests/test_Noise2VoidDataWrapper.py
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ def create_data(y_shape):
Y = np.random.rand(*y_shape)
return Y

def random_neighbor_withCP_uniform(patch, coord, dims):
def random_neighbor_withCP_uniform(patch, coord, dims, structN2Vmask=None):
return np.random.rand(1, 1)

def _getitem2D(y_shape):
Expand Down
Loading