Skip to content

Latest commit

 

History

History
executable file
·
276 lines (235 loc) · 27.8 KB

FileIO.md

File metadata and controls

executable file
·
276 lines (235 loc) · 27.8 KB
Raw

§1. Binary IO

  • FortranFiles.jl :: Read and write "unformatted" Fortran data, i.e. binary data interspersed with metadata about data records.
  • HexEdit.jl :: Edit and display binary file data in hexadecimal format.
  • LargeColumns.jl :: Handle large columns (vectors of equal length) with bits types in Julia using mmap - a thin wrapper for mmapped binary data, with a few sanity checks and convenience functions.
  • StrPack.jl :: For encoding and decoding binary data streams and there is some documentation at readthedocs.org.

§2. Data Formats

Libraries for Data serialization and (human readable) file formats.

  • AIDA.jl :: This package implements simple readers for legacy AIDA files.
  • BEncode.jl :: A Julia library for BEncode metadata files used for encoding BitTorrent storage and transmission of loosely structured data.
  • BDF.jl :: Module to read Biosemi BDF files with the Julia programming language.
    • BDF.jl fork by @codles :: A fork of BDF.jl that has started to add support for EDF files.
  • BGZF.jl :: A Julia package to read/write BGZF compressed files.
  • Brukerfile.jl :: Read Bruker data files.
  • BufferedStreams.jl :: Fast composable IO streams.
  • COFF.jl :: A julia implementation of the PE/COFF file format.
  • DWARF.jl :: Julia Package for parsing the DWARF file format.
  • ELF.jl :: Julia Package for working with ELF files.
  • FastaIO.jl :: Utilities to read/write FASTA format files in Julia.
  • FlatBuffers.jl :: A package to support Julia code generated by Google's flatbuffers compiler flatc.
  • Grisu.jl :: Port of Double-Conversion library to native Julia.
  • IniFile.jl :: Reading and writing Windows-style INI files.
  • MachO.jl :: An implementation of the MachO file format.
  • MNIST.jl :: Tools for working with the MNIST data set. This package provides access to the classic MNIST data set of handwritten digits that has been used as a testbed for new machine learning methods. The MNIST data set is included with the package, downloaded into their original IDX format and are stored in the data/ directory.
  • NeuralynxNCS.jl :: A Julia module for reading Neuralynx NCS files.
  • NetCDF.jl :: NetCDF support for a high-level and a medium-level interface for writing and reading netcdf files, for the Julia programming language.
  • NPZ.jl :: Julia package that provides support for reading and writing Numpy .npy and .npz files.
  • ObjFileBase.jl :: Common functionality for Object Files.
  • OIFITS.jl :: Support for OI-FITS (optical interferometry data format).
  • OpenSlide.jl :: OpenSlide bindings for Julia.
  • PdbTool :: An object-oriented Julia tool to parse PDB files and work with them.
  • PList.jl :: A module for reading and writing OS X plist in ASCII format. The binary and XML format is not supported presently.
  • PLX.jl :: Julia module for reading Plexon PLX files.
  • HarwellBoeing.jl :: A reader for matrices and supplementary data written in the Harwell-Boeing format.
  • SASLib.jl :: Provide a Julia library for reading SAS7BDAT data sets.
  • Shapefile.jl :: Parsing .shp files in Julia.
  • Silo.jl :: Wrapper for the Silo file format.
  • XPT.jl :: The XPT package reads SAS® software transport files and converts SAS software datasets to DataFrames.

§2.1. [Avro](https://en.wikipedia.org/wiki/Apache_Avro]

  • Avro.jl :: Julia implementation of the Apache Avro data standard. It provides convenient APIs for reading/writing data directly in the avro format, or as schema-included object container files.

§2.2. BSON

  • BSON.jl :: A Julia package for working with the Binary JSON serialisation format, that can be used as a general store for Julia data structures.
  • BSONify.jl :: Convert just about any data structure - including primitive types, arbitrary composite types defined by other modules or your own data structures, nested and recursive data structures, complicated parametrised types, and even type unions - to clean and simple BSON and back again. Save and load to/from files or a MongoDB.

JSON

  • Json2.jl :: JSON encoder/decoder on Julia.
  • JSONTables.jl :: JSON + Tables.jl
  • LazyJSON.jl :: LazyJSON is an interface for reading JSON data in Julia programs.
  • uJSON.jl :: JSON library for Julia using ultraJSON.

ODS

  • OdsIO.jl :: Open Document Format Spreadsheet (ODS) I/O for Julia using the python ezodf module.

PDF

  • PDFIO.jl :: PDF Reader Library for native Julia.
    • LabelNumerals.jl : Numbers used as labels like page number like alpha, roman numerals or prefixed like A-1,2 etc.

Silo

Tabular Data

  • CSV.jl by @JuliaData :: Utility library for working with CSV and other delimited files in the Julia programming language.
  • CSVFiles.jl :: FileIO.jl integration for CSV files.
  • DataRead.jl :: Read files from Stata, SAS, and SPSS.
  • StatFiles.jl :: FileIO.jl integration for Stata, SPSS, and SAS files
  • Feather.jl :: Julia library for working with feather-formatted files.
  • JuliaDB.jl :: A package based on Dagger and IndexedTables, that provides a "distributed array"-like data model for working with large persistent data sets. This all-Julia, end-to-end tool can load multi-dimensional datasets quickly and incrementally, index the data and perform filter, aggregate, sort and join operations, then save results and load them efficiently later using Julia's built-in parallelism to fully utilize any machine or cluster.
  • LAJuliaUtils.jl :: Utility functions for Julia, mainly dataframes operations.
  • MAT.jl :: Julia module for reading MATLAB files.
  • Parquet.jl :: Julia implementation of parquet columnar file format reader and writer.
  • ParquetFiles.jl :: This package provides load support for Parquet files under the FileIO.jl package.
  • Tables.jl :: This package provides four useful interface functions for working with tabular data in a variety of formats. The discourse announcment:!
  • TableWidgets.jl :: Interactive widgets to work with tabular data in Julia.
  • ReadWriteDlm2 :: CSV IO. Works like readdlm/writedlm, but using decimal comma by default. Additional supporting Date, DateTime, Time, Complex, Missing and Rational types.
  • Schemata.jl :: Schema (specification of a data set) for tabular data sets in Julia.

XDMF

  • XDMF.jl :: A standardized method to exchange scientific data between High Performance Computing codes and tools lead to the development of the eXtensible Data Model and Format (XDMF)

XML

  • EodDataTestXml.jl :: Test files for EodData.jl.
  • EzXML.jl :: XML/HTML handling tools for primates.
  • LibExpat.jl :: A Julia wrapper for libexpat.
  • LightXML.jl :: A light-weight Julia package for XML based on libxml2.
  • MusicXML.jl :: A parser for MusicXML files, designed to be used with the MXMLSynth library.
  • mzXML.jl :: A Julia package for reading mass spectrometry mzXML files.
  • SpimRegXML.jl.
  • XML.jl :: A libXML2 wrapper for Julia language.
  • XML2JSON.jl :: This package implements a fairly simplistic XML-to-JSON conversion.
  • XMLconvert.jl :: Functions for converting XML documents to other formats.
  • XMLDict.jl :: This package implements a simple Associative interface for XML documents.
  • XMLRPC.jl :: Send and recieve XML-RPCs with Julia.
  • XmlToDict.jl :: Julia module that makes working with XML feel like you are working with JSON (inspired by xmltodict).

HTML-XML

  • Patchwork.jl :: A library for representing the DOM in Julia. It supports element creation, diff computation and browser-side patching for efficient re-rendering.

§3. Data Type.

  • Arrow.jl :: Julia implementation of the Apache Arrow data standard.
  • ASCIIByte.jl :: Julia package to deal with Characters of 8 bits.
  • AutoFormat.jl
  • BigRationals.jl :: A BigRational package for Julia using GMP.
  • CBOR.jl :: A Concise Binary Object Representation (RFC 7049) Julia package for working with the CBOR data format, providing straightforward encoding and decoding for Julia types.
  • ChainedVectors.jl :: Few utility types over Julia Vector type.
  • Codecs.jl :: Common data encoding algorithms.
  • CustomUnitRanges.jl :: Package-specific AbstractUnitRange types for julia.
  • DanaTypes.jl :: Types for continuous variables or parameters.
  • DictViews.jl :: KeysView and ValuesView types for dynamic low-overhead views into the entries of dictionaries.
  • DotPlusInheritance.jl :: Expression parser that simulates type inheritance.
  • FixedPointNumbers.jl :: This library exports fixed-point number types. A fixed-point number represents a fractional, or non-integral, number. In contrast with the more widely known floating-point numbers, fixed-point numbers have a fixed number of digits (bits) after the decimal (radix) point. They are effectively integers scaled by a constant factor.
  • FixedSizeStrings.jl :: A type for efficiently storing short strings of known size.
  • Format.jl :: This Julia package provides C and Python-style types and functions formatting support.
  • Formatting.jl :: A Julia package to provide Python-like formatting support.
  • FreeTypeAbstraction.jl :: A Julian abstraction layer over FreeType.jl.
  • frange
  • FunctionalData.jl :: Functional, efficient data manipulation framework.
  • ImagineFormat.jl :: Read .imagine file formats in Julia.
  • jenks.jl
  • julia-prettyshow :: A module to provide simple pretty printing facilities with base functionality for indentation etc, and a pshow (pretty show) implementation for julia ASTs.
  • LaTeXStrings.jl :: This is a small package to make it easier to type LaTeX equations in string literals in the Julia language.
  • LHEF.jl :: Quick and dirty implementation of the Les Houches Event Format, for particle Physics, in terms of Fortran commonblocks where the information is embedded in a minimal XML-style structure.
  • MonkeyString.jl :: Fast string implementation inspired by SpiderMonkey.
  • MPFI.jl :: A MPFI wrapper for Julia.
  • MutableStrings.jl :: Mutable string types for Julia.
  • MUtils.jl :: channels(), tspaces(), kvspaces() and more.
  • Netpbm.jl :: Load and save Netpbm images in Julia. The Netpbm format is a graphics format using the extensions portable pixmap format (PPM), the portable graymap format (PGM) and the portable bitmap format (PBM), collectively referred as the portable anymap format (PNM).
  • Parsers.jl.
  • ResultTypes.jl :: A Result type for Julia—it's like Nullables for Exceptions.
  • ReTerms.jl :: Package providing abstract random-effects terms and specific types.
  • Ratios.jl :: Faster Rational-like types for Julia.
  • RDF.jl :: RDF package for working with RDF Graphs in Julia. Supports serialization as RDF N-Triples, RDF N-Quads and Turtle. Julia package mirror
  • Scalar.jl :: Scalar Types.
  • Sexpr.jl :: A program to port clojure-like s-expression syntax to and from Julia.
  • ShortStrings.jl :: A fast and efficient string format implementation for storing strings of size less than 15 bytes.
  • SimpleStructs.jl :: Easy to use struct definition with defaults and value constraints, as well as auto-defined user-friendly constructors.
  • StringDistances.jl :: String Distances in Julia.
  • StringLiterals.jl :: Implement improved string literals with Swift-style syntax for interpolation, hex, & unicode characters, plus C & Python style formatting and Unicode, HTML, LaTeX, and Emoji entities.
  • StringUtils.jl :: String utilities for Julia, Swift-like interpolation/quoting, better performing versions of string functions, etc.
  • TexExtensions.jl :: Tex Pretty printing of Julia Base types.
  • TypeSortedCollections.jl :: It provides the TypeSortedCollection type, which can be used to store type-heterogeneous data in a way that allows operations on the data to be performed in a type-stable manner.
  • Units.jl :: Infrastructure for handling physical units for the Julia programming language.
  • Unitful.jl :: A Julia package for physical units.
  • VersionParsing.jl :: flexible VersionNumber parsing in Julia.
  • WeakRefStrings.jl :: a minimal String type for Julia that allows for efficient string representation and transfer.
  • YAML.jl :: A flexible data serialization format that is designed to be easily read and written by human beings.

Composite Data Types

  • DictWrappers.jl :: Wrap any Julia composite type in an Associative interface.
  • EMLTranslator.jl :: Adds Inheritance to julia composite type.
  • ExtractMacro.jl :: Provides a convenience Julia macro to extract fields from composite types.
  • Modifyfield.jl :: It creates functions to modify immutable fields of a composite type inside a container.

Type Parameters

Resources

§4. Document Processors.

  • ConfParser.jl :: Package for parsing configuration files utilizing ini, http, and simple configuration syntaxes.
  • ExcelReaders.jl :: A package that provides functionality to read Excel files.
  • JuliaReport.jl :: A scientific report generator/literate programming tool for Julia based on Pweave and resembles Knitr and Sweave. JuliaReport relies on the Python package Pweave for document parsing and formatting.
  • Taro.jl :: can process office documents in Julia.

§4.1. Document Generator

  • Judo.jl :: is a Julia document generator, which takes documents written in pandoc markdown and converts them into html, but differs from general purpose markdown tools in a few ways.
  • HelpTestbed.jl :: package is for exploring options for help when you add a Julia package - when used from the REPL, the @help macro fetches, say, the signature of a function call, which can be used to find the package.
  • Publish.jl :: A package for publishing code - uses Latex to create a pdf with your code and output.
  • Report.jl :: A Markdown report writer for Julia.
  • VerTeX.jl :: Typeset scattered graph data rewriter based on LaTeX nodes.

§4.2. LaTeX.

  • Expr2LaTeX.jl :: Take a julia Expr and render it as LaTeX.
  • jlcode :: A latex package for displaying Julia code with the listings package.
  • LaTeX.jl :: Create LaTeX documents from within Julia, including image handling.
  • Latexify.jl :: Convert julia objects to LaTeX equations, arrays or other environments.
  • LatexPrint.jl :: Julia objects in a form suitable for LaTeX mathematics mode.
  • LaTeXStrings.jl :: This is a small package to make it easier to type LaTeX equations in string literals in the Julia language, written by Steven G. Johnson.
  • LaTeXTools.jl :: Tools for automatic manipulation and management of LaTeX Documents.

§5. General IO

  • FileIO.jl :: general dispatcher for save and load, support for recognizing magic bytes and file-extensions.
  • MapD.jl :: A julia library for conveniently inserting data to MapD.
  • SerialPorts.jl :: SerialPort IO streams in Julia.
  • StructIO.jl :: Experimental new implementation of StrPack.jl-like functionality.
  • ULID.jl :: Generate Universally Unique Lexicographically Sortable Identifiers (ULIDs) in Julia.

§6. Graphics Format

  • AndorSIF.jl :: This implements support for reading Andor SIF image files in the Julia programming language. Note: The Andor .SIF format changes with each version.
  • DICOM.jl :: DICOM interface for the Julia language.
  • FITSIO.jl :: Flexible Image Transport System (FITS) support for Julia.
  • ImageMagick.jl :: Thin Wrapper for the library ImageMagick, for general-purpose image saving and loading.
  • NIfTI.jl :: Julia module for reading NIfTI MRI files.
  • NRRD.jl :: Julia support for the Nearly Raw Raster Data (NRRD) image file format
  • Netpbm.jl :: Julia support for Netpbm (.ppm, .pgm, .pbm).
  • OMETIFF.jl :: I/O operations for OME-TIFF files in Julia with a focus on correctness.
  • QuartzImageIO.jl :: Thin wrapper around OSX's native image format saving and loading
  • TIFF.jl :: is a TIFF image file support for Julia.
  • TiffImages.jl :: This package aims to be a fast, minimal, and correct TIFF I/O, reader and writer, written in Julia.
  • VideoIO.jl :: A wrapper around libav/ffmpeg libraries, which are the defacto open-source libraries for video IO. The library offers an easy way to open video files or a camera and read sequences of images, as either arrays, or optionally as Image objects, using the Images package, has been developed on Linux, and the installation and functionality has been minimally tested on Macs, but not yet on Windows.

§7. Scientific Data Formats

  • HDF5.jl :: Lib to read HDF5 format files, a widely-used file format for general data.
  • HDF5Logger.jl :: Allows logging individual frames of data to an HDF5 file over time.
  • JLD.jl :: High-level interface to HDF5 for saving and loading Julia variables & types
  • JLD2 :: Next-generation successor to JLD.