Skip to content

Commit

Permalink
wrapper function added for lazy people (ie me)
Browse files Browse the repository at this point in the history
also rebuilt demo data to match demo extdata, tweaked variable name in
`n38_import()`, updated readme
  • Loading branch information
obrl_soil committed May 10, 2018
1 parent c1af191 commit 1405319
Show file tree
Hide file tree
Showing 11 changed files with 141 additions and 31 deletions.
4 changes: 2 additions & 2 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
Package: em38
Type: Package
Title: Process N38 binary files from EM38-MK2 sensors
Version: 0.0.0.9000
Version: 0.0.0.9001
Authors@R: person("Lauren", "O'Brien", email = "[email protected]", role = c('aut', 'cre'))
Description: Interprets and decodes the '.N38' file format used by the
Geonics EM38-MK2 electromagnetic sensing instrument, as described in its
Geonics EM38-MK2 ground conductivity meter, as described in its
manual. Conversion to '.M38' format or an 'sf' point dataset is supported.
This package offers an alternative to the existing format conversion
software, which lacks a command-line interface and thus cannot be easily
Expand Down
1 change: 1 addition & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ export(n38_chunk)
export(n38_decode)
export(n38_import)
export(n38_to_m38)
export(n38_to_points)
importFrom(dplyr,bind_rows)
importFrom(dplyr,group_by)
importFrom(dplyr,lag)
Expand Down
5 changes: 5 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
# v. 0.0.0.9001

* wrapper function added - `n38_to_points()` goes from on-disk file to spatial points in one line.
* rebuilt demo data to match demo extdata
* variable name fix in `n38_import()`

# v. 0.0.0.9000

Expand Down
10 changes: 5 additions & 5 deletions R/import_export.R
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
#' Import EM38 data
#'
#' This function reads N38 binary files into R and does some minimal pre-processing.
#' @param x A file path pointing to a valid *.N38 file, produced by a Geonics EM38-MK2 conductivity
#' sensor connected to an Allegra CX datalogger (and optionally, a GPS device).
#' @param path A file path pointing to a valid *.N38 file, produced by a Geonics EM38-MK2 conductivity
#' sensor connected to an Allegra CX or Archer datalogger (and optionally, a GPS device).
#' @return A matrix with n rows and 25 columns, containing raw bytes.
#' @examples
#' n38_mat <- n38_import(system.file("extdata", "em38_demo.n38", package = "em38"))
#' @export
#'
n38_import <- function(x = NULL) {
n38_import <- function(path = NULL) {

n38_con <- file(x, open = 'rb')
n38_raw <- readBin(n38_con, what = raw(), n = file.size(x))
n38_con <- file(path, open = 'rb')
n38_raw <- readBin(n38_con, what = raw(), n = file.size(path))
close(n38_con)

n38_mat <- matrix(n38_raw, ncol = 26, byrow = TRUE)
Expand Down
43 changes: 40 additions & 3 deletions R/spatialise.R
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ get_loc_data <- function(block = NULL) {

}

#' Spatialise data
#' Spatialise EM38 data
#'
#' This function processes a decoded N38 record into a point spatial dataset.
#' @param n38_decoded Nested list output by n38_decode
Expand All @@ -38,7 +38,9 @@ get_loc_data <- function(block = NULL) {
#' Horizontal data, never both.
#' @return An sf data frame with sfc_POINT geometry. WGS84 projection. If the n38_decoded object
#' contains more than one survey line, a list of sf objects is returned - one for each line.
#' @note Input n38_decoded object must be of survey type 'GPS' and record type 'auto'.
#' @note Input n38_decoded object should be of survey type 'GPS' and record type 'auto'. If not, the
#' function will fail gracefully by returning a list of reasons why the data could not be
#' converted to points.
#' @examples
#' data('n38_demo')
#' n38_chunks <- n38_chunk(n38_demo)
Expand All @@ -64,7 +66,11 @@ em38_spatial <- function(n38_decoded = NULL,
readings <- n38_decoded[[i]][['reading_data']]

# filter readings to chosen dipole mode
readings <- readings[readings$mode == out_mode, ]
readings <- if(out_mode %in% c('Vertical', 'Horizontal')) {
readings[readings$mode == out_mode, ]
} else {
return('Please check the value of out_mode.')
}

# if no readings of the chosen out_mode exist in this sl,
if(nrow(readings) == 0) {
Expand Down Expand Up @@ -217,3 +223,34 @@ em38_spatial <- function(n38_decoded = NULL,
}

}

#' Import and convert N38 logfiles to points
#'
#' This is a wrapper function that processes a raw on-disk N38 file into an sf point spatial
#' dataset.
#' @param path A file path pointing to a valid *.N38 file, produced by a Geonics EM38-MK2
#' conductivity sensor connected to an Allegra CX or Archer datalogger (and optionally, a GPS
#' device).
#' @param hdop_filter Numeric, discard GPS data where the Horizontal Dilution of Precision is
#' greater than this number. Defaults to 3 metres. Set to NULL to keep all readings.
#' @param out_mode Character, em38 dipole mode. Output dataset can only contain Vertical or
#' Horizontal data, never both.
#' @return An sf data frame with sfc_POINT geometry. WGS84 projection. If the n38_decoded object
#' contains more than one survey line, a list of sf objects is returned - one for each line.
#' @note Input file should be of survey type 'GPS' and record type 'auto'. If not, the
#' function will fail gracefully by returning reasons why the data could not be
#' converted to points.
#' @examples
#' vert_points <-
#' n38_to_points(path = system.file("extdata", "em38_demo.n38", package = "em38"),
#' hdop_filter = 3, out_mode = 'Vertical')
#' @export
n38_to_points <- function(path = NULL, hdop_filter = 3,
out_mode = c('Vertical', 'Horizontal')) {
mat <- n38_import(path)
chnk <- n38_chunk(mat)
dec <- n38_decode(chnk)

em38_spatial(n38_decoded = dec, hdop_filter = hdop_filter, out_mode = out_mode)

}
30 changes: 22 additions & 8 deletions README.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -20,35 +20,49 @@ em38 translates *.n38 binary files generated by the [Geonics EM38-MKII ground co

## Installation

Install development versions from github with
Install from github with

```{r 'instalation', eval = FALSE}
```{r 'installation', eval = FALSE}
library(devtools)
install_github("obrl-soil/em38")
```

This package is in early development phase. Improving its reliability requires access to a wide variety of test datasets, so if you have an *.n38 file that fails to decode correctly, please consider sending it to me. I have been able to test decoding on around 50 sample files from two different EM38-MKII devices, but could always use more. The device and its logger have a large number of possible setting combinations, and have not seen all of them. Additionally, a third-party GPS needs to be attached to the device and its data-logger, and GPS output data are notoriously variable by brand and model.
This package is in early development phase. Improving its reliability requires access to a wide variety of test datasets, so if you have an *.n38 file that fails to decode correctly, please consider sending it to me. I have been able to test decoding on around 50 sample files from two different EM38-MKII devices, but could always use more. The device and its logger have a large number of possible setting combinations, and I have not seen all of them. Additionally, a third-party GPS needs to be attached to the device and its data-logger, and GPS output data are notoriously variable by brand and model.

## Usage

em38 is accompanied by a demo dataset:

```{r example}
```{r 'example', message=FALSE}
library(em38)
# all-in-one wrapper function:
em38_sf <- n38_to_points(path = system.file("extdata", "em38_demo.n38", package = "em38"),
hdop_filter = 3, out_mode = 'Vertical')
# Plot spatialised output (calibrated conductivity for coil separation 0.5m)
plot(em38_sf['cond_05'], pch = 20)
```

Its surprisingly difficult to walk in a straight line across a paddock :no_mouth:

If you want to look at the intermediate data more closely,

```{r 'longwayround'}
# import binary file as raw() type matrix
n38_mat <- n38_import(system.file("extdata", "em38_demo.n38", package = "em38"))
# break matrix into sections according to file spec
n38_chunks <- n38_chunk(n38_mat)
# decode matrix chunks into useable data
n38_decoded <- n38_decode(n38_chunks)
# output an m38 file (comparable to official decode)
m38_example <- n38_to_m38(n38_decoded)
# write to file as e.g. write(m38_example, paste0('m38_from_R_', Sys.Date(), '.m38'))
# spatialise the decoded data
em38_sf <- em38_spatial(n38_decoded, out_mode = 'Vertical')
em38_sf <- em38_spatial(n38_decoded, hdop_filter = 3, out_mode = 'Vertical')
# Plot spatialised output (calibrated conductivity for coil separation 0.5m)
plot(em38_sf['cond_05'], pch = 20)
```

***
29 changes: 21 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,14 @@ em38 translates \*.n38 binary files generated by the [Geonics EM38-MKII ground c
Installation
------------

Install development versions from github with
Install from github with

``` r
library(devtools)
install_github("obrl-soil/em38")
```

This package is in early development phase. Improving its reliability requires access to a wide variety of test datasets, so if you have an \*.n38 file that fails to decode correctly, please consider sending it to me. I have been able to test decoding on around 50 sample files from two different EM38-MKII devices, but could always use more. The device and its logger have a large number of possible setting combinations, and have not seen all of them. Additionally, a third-party GPS needs to be attached to the device and its data-logger, and GPS output data are notoriously variable by brand and model.
This package is in early development phase. Improving its reliability requires access to a wide variety of test datasets, so if you have an \*.n38 file that fails to decode correctly, please consider sending it to me. I have been able to test decoding on around 50 sample files from two different EM38-MKII devices, but could always use more. The device and its logger have a large number of possible setting combinations, and I have not seen all of them. Additionally, a third-party GPS needs to be attached to the device and its data-logger, and GPS output data are notoriously variable by brand and model.

Usage
-----
Expand All @@ -23,22 +23,35 @@ em38 is accompanied by a demo dataset:

``` r
library(em38)

# all-in-one wrapper function:
em38_sf <- n38_to_points(path = system.file("extdata", "em38_demo.n38", package = "em38"),
hdop_filter = 3, out_mode = 'Vertical')


# Plot spatialised output (calibrated conductivity for coil separation 0.5m)
plot(em38_sf['cond_05'], pch = 20)
```

![](README-example-1.png)

Its surprisingly difficult to walk in a straight line across a paddock :no\_mouth:

If you want to look at the intermediate data more closely,

``` r
# import binary file as raw() type matrix
n38_mat <- n38_import(system.file("extdata", "em38_demo.n38", package = "em38"))
# break matrix into sections according to file spec
n38_chunks <- n38_chunk(n38_mat)
# decode matrix chunks into useable data
n38_decoded <- n38_decode(n38_chunks)

# output an m38 file (comparable to official decode)
m38_example <- n38_to_m38(n38_decoded)
# write to file as e.g. write(m38_example, paste0('m38_from_R_', Sys.Date(), '.m38'))

# spatialise the decoded data
em38_sf <- em38_spatial(n38_decoded, out_mode = 'Vertical')

# Plot spatialised output (calibrated conductivity for coil separation 0.5m)
plot(em38_sf['cond_05'], pch = 20)
em38_sf <- em38_spatial(n38_decoded, hdop_filter = 3, out_mode = 'Vertical')
```

![](README-example-1.png)
------------------------------------------------------------------------
Binary file modified data/n38_demo.rda
Binary file not shown.
6 changes: 4 additions & 2 deletions man/em38_spatial.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions man/n38_import.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

38 changes: 38 additions & 0 deletions man/n38_to_points.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit 1405319

Please sign in to comment.