forked from Explore-AI/Public-Data
-
Notifications
You must be signed in to change notification settings - Fork 0
/
output_log.txt
33 lines (33 loc) · 10.5 KB
/
output_log.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
26 Jun 2021 05:43:01,353 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.taskrunner.TaskPoller: Executing: amazonaws.datapipeline.activity.ShellCommandActivity@4ad5e1aa
26 Jun 2021 05:43:01,713 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA private.com.amazonaws.services.datapipeline.factory.S3ClientFactory: Returning cached AmazonS3Client for the region [eu-west-1]
26 Jun 2021 05:43:02,130 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.connector.staging.S3Helper: $ Going to decorating s3 client with region
26 Jun 2021 05:43:02,131 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.connector.staging.S3Helper: $ decorating s3 client with region
26 Jun 2021 05:43:02,208 [WARN] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.retrier.S3Retrier: Failed to invoke object Moved Permanently (Service: Amazon S3; Status Code: 301; Error Code: 301 Moved Permanently; Request ID: 795FWM2SRCX83GXP)
26 Jun 2021 05:43:02,208 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA private.com.amazonaws.services.datapipeline.factory.S3ClientFactory: Creating new AmazonS3Client for region [eu-west-1]
26 Jun 2021 05:43:02,704 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.connector.staging.S3Helper: Begin Downloading directory from S3 Path:s3://de-course-book-case/ to output/staging/df-03282251I0S1AZJKI7SA_input1_58a44ae9b90d481586c8ff311a00ff1a
26 Jun 2021 05:43:02,705 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.connector.staging.S3Helper: $ decorating s3 client with region
26 Jun 2021 05:43:02,712 [WARN] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.retrier.S3Retrier: Failed to invoke object Moved Permanently (Service: Amazon S3; Status Code: 301; Error Code: 301 Moved Permanently; Request ID: 795BNMTM8DVZX1F9)
26 Jun 2021 05:43:02,712 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA private.com.amazonaws.services.datapipeline.factory.S3ClientFactory: Returning cached AmazonS3Client for the region [eu-west-1]
26 Jun 2021 05:43:02,800 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.connector.staging.S3Helper: Begin Downloading files from S3 Path:s3://de-course-book-case/ to output/staging/df-03282251I0S1AZJKI7SA_input1_58a44ae9b90d481586c8ff311a00ff1a
26 Jun 2021 05:43:02,800 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.connector.staging.S3Helper: Local File Relative compared to Input Root Path:s3://de-course-book-case/ is s3_data.txt
26 Jun 2021 05:43:02,800 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.connector.staging.S3Helper: Begin Downloading S3 file s3://de-course-book-case/s3_data.txt to /mnt/taskRunner/output/staging/df-03282251I0S1AZJKI7SA_input1_58a44ae9b90d481586c8ff311a00ff1a/s3_data.txt
26 Jun 2021 05:43:02,806 [WARN] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.retrier.S3Retrier: Failed to invoke object Moved Permanently (Service: Amazon S3; Status Code: 301; Error Code: 301 Moved Permanently; Request ID: 7958GGXTRPS407MN)
26 Jun 2021 05:43:02,806 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA private.com.amazonaws.services.datapipeline.factory.S3ClientFactory: Creating new S3 Transfer Manager for region [eu-west-1]
26 Jun 2021 05:43:02,806 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA private.com.amazonaws.services.datapipeline.factory.S3ClientFactory: Returning cached AmazonS3Client for the region [eu-west-1]
26 Jun 2021 05:43:03,485 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.connector.staging.S3Helper: Completed Downloading files from S3 Path:s3://de-course-book-case/ to output/staging/df-03282251I0S1AZJKI7SA_input1_58a44ae9b90d481586c8ff311a00ff1a
26 Jun 2021 05:43:03,485 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.connector.staging.S3Helper: Completed Downloading directory from S3 Path:s3://de-course-book-case/ to output/staging/df-03282251I0S1AZJKI7SA_input1_58a44ae9b90d481586c8ff311a00ff1a
26 Jun 2021 05:43:03,486 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.objects.CommandRunner: Executing command: grep -rc "Ahab" ${INPUT1_STAGING_DIR}/* > ${OUTPUT1_STAGING_DIR}/output.txt
26 Jun 2021 05:43:03,494 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.objects.CommandRunner: configure ApplicationRunner with stdErr file: output/logs/df-03282251I0S1AZJKI7SA/ShellCommandActivityObj/@ShellCommandActivityObj_2021-06-26T05:39:15/@ShellCommandActivityObj_2021-06-26T05:39:15_Attempt=1/StdError and stdout file :output/logs/df-03282251I0S1AZJKI7SA/ShellCommandActivityObj/@ShellCommandActivityObj_2021-06-26T05:39:15/@ShellCommandActivityObj_2021-06-26T05:39:15_Attempt=1/StdOutput
26 Jun 2021 05:43:03,494 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.objects.CommandRunner: Executing command: output/tmp/df-03282251I0S1AZJKI7SA-e07943e6f50f47fda4a2e45d78fe89c6/ShellCommandActivityObj20210626T053915Attempt1_command.sh with env variables :{INPUT1_STAGING_DIR=/mnt/taskRunner/output/staging/df-03282251I0S1AZJKI7SA_input1_58a44ae9b90d481586c8ff311a00ff1a, OUTPUT1_STAGING_DIR=/mnt/taskRunner/output/staging/df-03282251I0S1AZJKI7SA_output1_bf21869d0d274ee29c8145fd2ea17c0d} with argument : null
26 Jun 2021 05:43:03,515 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.connector.staging.S3Helper: Begin Uploading local directory:output/staging/df-03282251I0S1AZJKI7SA_output1_bf21869d0d274ee29c8145fd2ea17c0d to S3 s3://de-course-book-case//2021-06-26-05-39-15
26 Jun 2021 05:43:03,515 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.connector.staging.S3Helper: Begin Upload single file to S3:s3://de-course-book-case//2021-06-26-05-39-15/output.txt
26 Jun 2021 05:43:03,521 [WARN] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.retrier.S3Retrier: Failed to invoke object Moved Permanently (Service: Amazon S3; Status Code: 301; Error Code: 301 Moved Permanently; Request ID: TX0VFH3SC3NRJMH1)
26 Jun 2021 05:43:03,521 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA private.com.amazonaws.services.datapipeline.factory.S3ClientFactory: Returning cached AmazonS3Client for the region [eu-west-1]
26 Jun 2021 05:43:03,555 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.connector.staging.S3Helper: Begin upload of file /mnt/taskRunner/output/staging/df-03282251I0S1AZJKI7SA_output1_bf21869d0d274ee29c8145fd2ea17c0d/output.txt to S3 paths3://de-course-book-case//2021-06-26-05-39-15/output.txt
26 Jun 2021 05:43:03,740 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.connector.staging.S3Helper: Completed upload of file /mnt/taskRunner/output/staging/df-03282251I0S1AZJKI7SA_output1_bf21869d0d274ee29c8145fd2ea17c0d/output.txt to S3 paths3://de-course-book-case//2021-06-26-05-39-15/output.txt
26 Jun 2021 05:43:03,740 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.connector.staging.S3Helper: Completed uploading of all files
26 Jun 2021 05:43:03,740 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.connector.staging.S3Helper: Completed upload of local dir output/staging/df-03282251I0S1AZJKI7SA_output1_bf21869d0d274ee29c8145fd2ea17c0d to s3://de-course-book-case//2021-06-26-05-39-15
26 Jun 2021 05:43:03,740 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.connector.staging.StageFromS3Connector: cleaning up directory /mnt/taskRunner/output/staging/df-03282251I0S1AZJKI7SA_input1_58a44ae9b90d481586c8ff311a00ff1a
26 Jun 2021 05:43:03,747 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.connector.staging.StageInS3Connector: cleaning up directory /mnt/taskRunner/output/staging/df-03282251I0S1AZJKI7SA_output1_bf21869d0d274ee29c8145fd2ea17c0d
26 Jun 2021 05:43:03,748 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.taskrunner.HeartBeatService: Finished waiting for heartbeat thread @ShellCommandActivityObj_2021-06-26T05:39:15_Attempt=1
26 Jun 2021 05:43:03,749 [INFO] (TaskRunnerService-resource:df-03282251I0S1AZJKI7SA_@EC2ResourceObj_2021-06-26T05:39:15-0) df-03282251I0S1AZJKI7SA amazonaws.datapipeline.taskrunner.TaskPoller: Work ShellCommandActivity took 0:2 to complete