Skip to content

Commit

Permalink
Move examples module and rename common-api to common (#12)
Browse files Browse the repository at this point in the history
  • Loading branch information
JackieTien97 authored Jan 6, 2024
1 parent 244cd0d commit ccb72f9
Show file tree
Hide file tree
Showing 25 changed files with 1,084 additions and 4 deletions.
8 changes: 7 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
/.idea/**
/.mvn/.gradle-enterprise/gradle-enterprise-workspace-id
/.mvn/wrapper/maven-wrapper.jar
**/target/**
/tsfile/test.tsfile

# intellij IDE files
**/*.iml
**/.idea/
**/*.log
**/*.ipr
**/*.iws
2 changes: 1 addition & 1 deletion common-api/pom.xml → common/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
<artifactId>tsfile-parent</artifactId>
<version>1.0.0-SNAPSHOT</version>
</parent>
<artifactId>common-api</artifactId>
<artifactId>common</artifactId>
<name>TsFile: Common API</name>
<profiles>
<profile>
Expand Down
77 changes: 77 additions & 0 deletions examples/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.tsfile</groupId>
<artifactId>tsfile-parent</artifactId>
<version>1.0.0-SNAPSHOT</version>
</parent>
<artifactId>tsfile-examples</artifactId>
<packaging>pom</packaging>
<name>TSFile: Examples</name>
<dependencies>
<dependency>
<groupId>org.apache.tsfile</groupId>
<artifactId>tsfile</artifactId>
<version>1.0.0-SNAPSHOT</version>
</dependency>
</dependencies>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>8</source>
<target>8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>check-dependencies</id>
<goals>
<goal>analyze-only</goal>
</goals>
<phase>verify</phase>
<configuration>
<skip>true</skip>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-enforcer-plugin</artifactId>
<configuration>
<!-- Managing the dependencies of all examples would not provide much, but be a lot of work -->
<skip>true</skip>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>
84 changes: 84 additions & 0 deletions examples/readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# Function
```
The example is to show how to write and read a TsFile File.
```
# Usage
## Dependencies with Maven

```
<dependencies>
<dependency>
<groupId>org.apache.tsfile</groupId>
<artifactId>tsfile</artifactId>
<version>1.0.0-SNAPSHOT</version>
</dependency>
</dependencies>
```


## Run TsFileWrite.java


This class is to show how to write a TsFile. It provided two ways:

The first one is using a JSON string for measurement(s). The JSON string is an array of JSON
objects(schema). The object must include the *measurement_id*, *datatype*, *encoding*, and
*compressor*.

An example JSON string is provided in the comments in
the method

public static void tsFileWriteWithJson() throws IOException,WriteProcessException
It uses this interface

public void addMeasurementByJson(JSONObject measurement) throws WriteProcessException
An alternative way is to add these measurements directly(manually) by the second interface:

public void addMeasurement(MeasurementSchema measurementSchema) throws WriteProcessException

The method

public static void tsFileWriteDirect() throws IOException,WriteProcessException
shows how to use that interface.

Note that the measurements in the two methods are the same therefore there output TsFile should also be identical.



## Run TsFileRead.java

This class is to show how to read TsFile file named "testDirect.tsfile".

The TsFile file "testDirect.tsfile" is generated from class TsFileWrite.

It generates the same TsFile(testDirect.tsfile and testWithJson.tsfile) file by two different ways

Run TsFileWrite to generate the testDirect.tsfile first

## Run TsFileSequenceRead.java

This class is to show the structure of a TsFile.

### Notice
 For detail, please refer to https://github.com/apache/iotdb/blob/master/tsfile/README.md.
33 changes: 33 additions & 0 deletions examples/src/main/java/org/apache/tsfile/Constant.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

package org.apache.tsfile;

public class Constant {

private Constant() {}

static final String SENSOR_1 = "sensor_1";
static final String SENSOR_2 = "sensor_2";
static final String SENSOR_3 = "sensor_3";

static final String DEVICE_PREFIX = "device_";
static final String DEVICE_1 = "root.sg.device_1";
static final String DEVICE_2 = "root.sg.device_2";
}
124 changes: 124 additions & 0 deletions examples/src/main/java/org/apache/tsfile/TsFileForceAppendWrite.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

package org.apache.tsfile;

import org.apache.tsfile.enums.TSDataType;
import org.apache.tsfile.file.metadata.enums.TSEncoding;
import org.apache.tsfile.fileSystem.FSFactoryProducer;
import org.apache.tsfile.read.common.Path;
import org.apache.tsfile.write.TsFileWriter;
import org.apache.tsfile.write.record.TSRecord;
import org.apache.tsfile.write.record.datapoint.DataPoint;
import org.apache.tsfile.write.record.datapoint.LongDataPoint;
import org.apache.tsfile.write.schema.MeasurementSchema;
import org.apache.tsfile.write.writer.ForceAppendTsFileWriter;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.File;
import java.io.IOException;
import java.nio.file.Files;

public class TsFileForceAppendWrite {

private static final Logger LOGGER = LoggerFactory.getLogger(TsFileForceAppendWrite.class);

public static void main(String[] args) throws IOException {
String path = "test.tsfile";
File f = FSFactoryProducer.getFSFactory().getFile(path);
if (f.exists()) {
Files.delete(f.toPath());
}

try (TsFileWriter tsFileWriter = new TsFileWriter(f)) {

// add measurements into file schema
for (int i = 0; i < 4; i++) {
tsFileWriter.registerTimeseries(
new Path(Constant.DEVICE_PREFIX + i),
new MeasurementSchema(Constant.SENSOR_1, TSDataType.INT64, TSEncoding.RLE));
tsFileWriter.registerTimeseries(
new Path(Constant.DEVICE_PREFIX + i),
new MeasurementSchema(Constant.SENSOR_2, TSDataType.INT64, TSEncoding.RLE));
tsFileWriter.registerTimeseries(
new Path(Constant.DEVICE_PREFIX + i),
new MeasurementSchema(Constant.SENSOR_3, TSDataType.INT64, TSEncoding.RLE));
}

// construct TSRecord
for (int i = 0; i < 100; i++) {
TSRecord tsRecord = new TSRecord(i, Constant.DEVICE_PREFIX + (i % 4));
DataPoint dPoint1 = new LongDataPoint(Constant.SENSOR_1, i);
DataPoint dPoint2 = new LongDataPoint(Constant.SENSOR_2, i);
DataPoint dPoint3 = new LongDataPoint(Constant.SENSOR_3, i);
tsRecord.addTuple(dPoint1);
tsRecord.addTuple(dPoint2);
tsRecord.addTuple(dPoint3);

// write TSRecord
tsFileWriter.write(tsRecord);
}
} catch (Exception e) {
LOGGER.error("meet error in TsFileWrite ", e);
}

// open the closed file with ForceAppendTsFileWriter

try (ForceAppendTsFileWriter fwriter = new ForceAppendTsFileWriter(f)) {
fwriter.doTruncate();
write(fwriter);
} catch (Exception e) {
LOGGER.error("ForceAppendTsFileWriter truncate or write error ", e);
}
}

private static void write(ForceAppendTsFileWriter fwriter) {
try (TsFileWriter tsFileWriter1 = new TsFileWriter(fwriter)) {
// add measurements into file schema
for (int i = 0; i < 4; i++) {
tsFileWriter1.registerTimeseries(
new Path(Constant.DEVICE_PREFIX + i),
new MeasurementSchema(Constant.SENSOR_1, TSDataType.INT64, TSEncoding.RLE));
tsFileWriter1.registerTimeseries(
new Path(Constant.DEVICE_PREFIX + i),
new MeasurementSchema(Constant.SENSOR_2, TSDataType.INT64, TSEncoding.RLE));
tsFileWriter1.registerTimeseries(
new Path(Constant.DEVICE_PREFIX + i),
new MeasurementSchema(Constant.SENSOR_3, TSDataType.INT64, TSEncoding.RLE));
}
// construct TSRecord
for (int i = 100; i < 120; i++) {
TSRecord tsRecord = new TSRecord(i, Constant.DEVICE_PREFIX + (i % 4));
DataPoint dPoint1 = new LongDataPoint(Constant.SENSOR_1, i);
DataPoint dPoint2 = new LongDataPoint(Constant.SENSOR_2, i);
DataPoint dPoint3 = new LongDataPoint(Constant.SENSOR_3, i);
tsRecord.addTuple(dPoint1);
tsRecord.addTuple(dPoint2);
tsRecord.addTuple(dPoint3);

// write TSRecord
tsFileWriter1.write(tsRecord);
}
} catch (Exception e) {
LOGGER.error("meet error in TsFileWrite ", e);
}
}
}
Loading

0 comments on commit ccb72f9

Please sign in to comment.