Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move examples module and rename common-api to common #12

Merged
merged 5 commits into from
Jan 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
/.idea/**
/.mvn/.gradle-enterprise/gradle-enterprise-workspace-id
/.mvn/wrapper/maven-wrapper.jar
**/target/**
/tsfile/test.tsfile

# intellij IDE files
**/*.iml
**/.idea/
**/*.log
**/*.ipr
**/*.iws
2 changes: 1 addition & 1 deletion common-api/pom.xml → common/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
<artifactId>tsfile-parent</artifactId>
<version>1.0.0-SNAPSHOT</version>
</parent>
<artifactId>common-api</artifactId>
<artifactId>common</artifactId>
<name>TsFile: Common API</name>
<profiles>
<profile>
Expand Down
77 changes: 77 additions & 0 deletions examples/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--

Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.

-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.tsfile</groupId>
<artifactId>tsfile-parent</artifactId>
<version>1.0.0-SNAPSHOT</version>
</parent>
<artifactId>tsfile-examples</artifactId>
<packaging>pom</packaging>
<name>TSFile: Examples</name>
<dependencies>
<dependency>
<groupId>org.apache.tsfile</groupId>
<artifactId>tsfile</artifactId>
<version>1.0.0-SNAPSHOT</version>
</dependency>
</dependencies>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>8</source>
<target>8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>check-dependencies</id>
<goals>
<goal>analyze-only</goal>
</goals>
<phase>verify</phase>
<configuration>
<skip>true</skip>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-enforcer-plugin</artifactId>
<configuration>
<!-- Managing the dependencies of all examples would not provide much, but be a lot of work -->
<skip>true</skip>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>
84 changes: 84 additions & 0 deletions examples/readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
<!--

Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.

-->

# Function
```
The example is to show how to write and read a TsFile File.
```
# Usage
## Dependencies with Maven

```
<dependencies>
<dependency>
<groupId>org.apache.tsfile</groupId>
<artifactId>tsfile</artifactId>
<version>1.0.0-SNAPSHOT</version>
</dependency>
</dependencies>
```


## Run TsFileWrite.java


This class is to show how to write a TsFile. It provided two ways:

The first one is using a JSON string for measurement(s). The JSON string is an array of JSON
objects(schema). The object must include the *measurement_id*, *datatype*, *encoding*, and
*compressor*.

An example JSON string is provided in the comments in
the method

public static void tsFileWriteWithJson() throws IOException,WriteProcessException
It uses this interface

public void addMeasurementByJson(JSONObject measurement) throws WriteProcessException
An alternative way is to add these measurements directly(manually) by the second interface:

public void addMeasurement(MeasurementSchema measurementSchema) throws WriteProcessException

The method

public static void tsFileWriteDirect() throws IOException,WriteProcessException
shows how to use that interface.

Note that the measurements in the two methods are the same therefore there output TsFile should also be identical.



## Run TsFileRead.java

This class is to show how to read TsFile file named "testDirect.tsfile".

The TsFile file "testDirect.tsfile" is generated from class TsFileWrite.

It generates the same TsFile(testDirect.tsfile and testWithJson.tsfile) file by two different ways

Run TsFileWrite to generate the testDirect.tsfile first

## Run TsFileSequenceRead.java

This class is to show the structure of a TsFile.

### Notice
 For detail, please refer to https://github.com/apache/iotdb/blob/master/tsfile/README.md.
33 changes: 33 additions & 0 deletions examples/src/main/java/org/apache/tsfile/Constant.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

package org.apache.tsfile;

public class Constant {

private Constant() {}

static final String SENSOR_1 = "sensor_1";
static final String SENSOR_2 = "sensor_2";
static final String SENSOR_3 = "sensor_3";

static final String DEVICE_PREFIX = "device_";
static final String DEVICE_1 = "root.sg.device_1";
static final String DEVICE_2 = "root.sg.device_2";
}
124 changes: 124 additions & 0 deletions examples/src/main/java/org/apache/tsfile/TsFileForceAppendWrite.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

package org.apache.tsfile;

import org.apache.tsfile.enums.TSDataType;
import org.apache.tsfile.file.metadata.enums.TSEncoding;
import org.apache.tsfile.fileSystem.FSFactoryProducer;
import org.apache.tsfile.read.common.Path;
import org.apache.tsfile.write.TsFileWriter;
import org.apache.tsfile.write.record.TSRecord;
import org.apache.tsfile.write.record.datapoint.DataPoint;
import org.apache.tsfile.write.record.datapoint.LongDataPoint;
import org.apache.tsfile.write.schema.MeasurementSchema;
import org.apache.tsfile.write.writer.ForceAppendTsFileWriter;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.File;
import java.io.IOException;
import java.nio.file.Files;

public class TsFileForceAppendWrite {

private static final Logger LOGGER = LoggerFactory.getLogger(TsFileForceAppendWrite.class);

public static void main(String[] args) throws IOException {
String path = "test.tsfile";
File f = FSFactoryProducer.getFSFactory().getFile(path);
if (f.exists()) {
Files.delete(f.toPath());
}

try (TsFileWriter tsFileWriter = new TsFileWriter(f)) {

// add measurements into file schema
for (int i = 0; i < 4; i++) {
tsFileWriter.registerTimeseries(
new Path(Constant.DEVICE_PREFIX + i),
new MeasurementSchema(Constant.SENSOR_1, TSDataType.INT64, TSEncoding.RLE));
tsFileWriter.registerTimeseries(
new Path(Constant.DEVICE_PREFIX + i),
new MeasurementSchema(Constant.SENSOR_2, TSDataType.INT64, TSEncoding.RLE));
tsFileWriter.registerTimeseries(
new Path(Constant.DEVICE_PREFIX + i),
new MeasurementSchema(Constant.SENSOR_3, TSDataType.INT64, TSEncoding.RLE));
}

// construct TSRecord
for (int i = 0; i < 100; i++) {
TSRecord tsRecord = new TSRecord(i, Constant.DEVICE_PREFIX + (i % 4));
DataPoint dPoint1 = new LongDataPoint(Constant.SENSOR_1, i);
DataPoint dPoint2 = new LongDataPoint(Constant.SENSOR_2, i);
DataPoint dPoint3 = new LongDataPoint(Constant.SENSOR_3, i);
tsRecord.addTuple(dPoint1);
tsRecord.addTuple(dPoint2);
tsRecord.addTuple(dPoint3);

// write TSRecord
tsFileWriter.write(tsRecord);
}
} catch (Exception e) {
LOGGER.error("meet error in TsFileWrite ", e);
}

// open the closed file with ForceAppendTsFileWriter

try (ForceAppendTsFileWriter fwriter = new ForceAppendTsFileWriter(f)) {
fwriter.doTruncate();
write(fwriter);
} catch (Exception e) {
LOGGER.error("ForceAppendTsFileWriter truncate or write error ", e);
}
}

private static void write(ForceAppendTsFileWriter fwriter) {
try (TsFileWriter tsFileWriter1 = new TsFileWriter(fwriter)) {
// add measurements into file schema
for (int i = 0; i < 4; i++) {
tsFileWriter1.registerTimeseries(
new Path(Constant.DEVICE_PREFIX + i),
new MeasurementSchema(Constant.SENSOR_1, TSDataType.INT64, TSEncoding.RLE));
tsFileWriter1.registerTimeseries(
new Path(Constant.DEVICE_PREFIX + i),
new MeasurementSchema(Constant.SENSOR_2, TSDataType.INT64, TSEncoding.RLE));
tsFileWriter1.registerTimeseries(
new Path(Constant.DEVICE_PREFIX + i),
new MeasurementSchema(Constant.SENSOR_3, TSDataType.INT64, TSEncoding.RLE));
}
// construct TSRecord
for (int i = 100; i < 120; i++) {
TSRecord tsRecord = new TSRecord(i, Constant.DEVICE_PREFIX + (i % 4));
DataPoint dPoint1 = new LongDataPoint(Constant.SENSOR_1, i);
DataPoint dPoint2 = new LongDataPoint(Constant.SENSOR_2, i);
DataPoint dPoint3 = new LongDataPoint(Constant.SENSOR_3, i);
tsRecord.addTuple(dPoint1);
tsRecord.addTuple(dPoint2);
tsRecord.addTuple(dPoint3);

// write TSRecord
tsFileWriter1.write(tsRecord);
}
} catch (Exception e) {
LOGGER.error("meet error in TsFileWrite ", e);
}
}
}
Loading