crash using neural strategy

[gianpaj]

System information

• Have I written custom code (as opposed to using zenbot vanilla):
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Mac
• Zenbot version (commit ref, or version): b1918f6

commit b1918f67f74c320e33179a3506fe3c725c3a6aa2 (HEAD -> unstable, origin/unstable, origin/HEAD)
Author: Maddcow <[email protected]>
Date:   Sat Feb 20 03:27:50 2021 -0500

    Added GDAX Sandbox option (#2653)

• Zenbot branch: unstable
• NodeJS version: v12.20.0
• Exact command to reproduce (include everything): node zenbot sim --strategy 'neural'
• Did I make any changes to conf-sample.js?:

conf.js

c.selector = process.env.ZENBOT_DEFAULT_SELECTOR || 'binance.BTC-USDT'

// enable trailing sell stop when reaching this % profit (0 to disable)
- c.profit_stop_enable_pct = process.env.ZENBOT_PROFIT_STOP_ENABLE_PCT || 0
+ c.profit_stop_enable_pct = process.env.ZENBOT_PROFIT_STOP_ENABLE_PCT || 10
// maintain a trailing stop this % below the high-water mark of profit
- c.profit_stop_pct = process.env.ZENBOT_PROFIT_STOP_PCT || 1
+ c.profit_stop_pct = process.env.ZENBOT_PROFIT_STOP_PCT || 4

Describe the problem

2021-02-16 14:59:59  49289.40 BTC-USDT   +0.05%     1751   53.56 +    0000.000000000     sell  0.00000 BTC  1000.00000000 USDT   +0.00%   -0.32%
<--- Last few GCs --->

[56436:0x102db8000]    49997 ms: Scavenge 2041.8 (2047.6) -> 2041.0 (2047.9) MB, 3.8 / 0.0 ms  (average mu = 0.078, current mu = 0.027) allocation failure
[56436:0x102db8000]    50008 ms: Scavenge 2042.2 (2047.9) -> 2041.5 (2048.6) MB, 4.1 / 0.0 ms  (average mu = 0.078, current mu = 0.027) allocation failure
[56436:0x102db8000]    50019 ms: Scavenge 2042.9 (2048.6) -> 2042.2 (2053.4) MB, 4.0 / 0.0 ms  (average mu = 0.078, current mu = 0.027) allocation failure


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x100a02f39]
Security context: 0x3932381008d1 <JSObject>
    1: getEncodingOps(aka getEncodingOps) [0x3932df9af4b9] [buffer.js:~676] [pc=0x4485144a9bf](this=0x393245c804b1 <undefined>,0x393276469f89 <String[#4]: utf8>)
    2: deserializeObject(aka deserializeObject) [0x39320ffd2251] [/Users/user/temp/zenbot/node_modules/bson/lib/bson/parser/deserializer.js:~43] [pc=0x4485146034e](this=0x393245c804b1 <undefined>,0...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x1011f5195 node::Abort() (.cold.1) [/Users/user/.nvm/versions/node/v12.20.0/bin/node]
 2: 0x1000a5a29 node::Abort() [/Users/user/.nvm/versions/node/v12.20.0/bin/node]
 3: 0x1000a5b8f node::OnFatalError(char const*, char const*) [/Users/user/.nvm/versions/node/v12.20.0/bin/node]
 4: 0x1001ee277 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/Users/user/.nvm/versions/node/v12.20.0/bin/node]
 5: 0x1001ee217 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/user/.nvm/versions/node/v12.20.0/bin/node]
 6: 0x100384db5 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/user/.nvm/versions/node/v12.20.0/bin/node]
 7: 0x10038660a v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [/Users/user/.nvm/versions/node/v12.20.0/bin/node]
 8: 0x10038308e v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/user/.nvm/versions/node/v12.20.0/bin/node]
 9: 0x100380e40 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/user/.nvm/versions/node/v12.20.0/bin/node]
10: 0x10037fe91 v8::internal::Heap::HandleGCRequest() [/Users/user/.nvm/versions/node/v12.20.0/bin/node]
11: 0x100345551 v8::internal::StackGuard::HandleInterrupts() [/Users/user/.nvm/versions/node/v12.20.0/bin/node]
12: 0x1006a9e0c v8::internal::Runtime_StackGuard(int, unsigned long*, v8::internal::Isolate*) [/Users/user/.nvm/versions/node/v12.20.0/bin/node]
13: 0x100a02f39 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_NoBuiltinExit [/Users/user/.nvm/versions/node/v12.20.0/bin/node]
[1]    56436 abort      node zenbot sim --strategy 'neural'

I was using this data:

node zenbot backfill --days 7

[127]

same on os x
running zenbot trade --paper
conf.js
c.output.api.on = false
error

<--- Last few GCs --->

[78961:0x102b90000]    77784 ms: Scavenge 4034.6 (4114.1) -> 4033.1 (4114.8) MB, 11.2 / 0.0 ms  (average mu = 0.298, current mu = 0.135) task
[78961:0x102b90000]    77811 ms: Scavenge 4035.4 (4114.8) -> 4034.4 (4134.8) MB, 14.5 / 0.0 ms  (average mu = 0.298, current mu = 0.135) task
[78961:0x102b90000]    82977 ms: Mark-sweep 4046.9 (4134.8) -> 4042.3 (4146.3) MB, 5059.9 / 0.0 ms  (average mu = 0.187, current mu = 0.055) task scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory
 1: 0x1000af220 node::Abort() [/usr/local/bin/node]
 2: 0x1000af396 node::OnFatalError(char const*, char const*) [/usr/local/bin/node]
 3: 0x1001ecccd v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 4: 0x1001ecc77 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 5: 0x10033157f v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/usr/local/bin/node]
 6: 0x10036b2d4 v8::internal::EvacuateNewSpaceVisitor::Visit(v8::internal::HeapObject, int) [/usr/local/bin/node]
 7: 0x10035ade9 void v8::internal::LiveObjectVisitor::VisitBlackObjectsNoFail<v8::internal::EvacuateNewSpaceVisitor, v8::internal::MajorNonAtomicMarkingState>(v8::internal::MemoryChunk*, v8::internal::MajorNonAtomicMarkingState*, v8::internal::EvacuateNewSpaceVisitor*, v8::internal::LiveObjectVisitor::IterationMode) [/usr/local/bin/node]
 8: 0x10035a9d7 v8::internal::FullEvacuator::RawEvacuatePage(v8::internal::MemoryChunk*, long*) [/usr/local/bin/node]
 9: 0x10035a6e6 v8::internal::Evacuator::EvacuatePage(v8::internal::MemoryChunk*) [/usr/local/bin/node]
10: 0x10036e045 v8::internal::PageEvacuationTask::ProcessItems() [/usr/local/bin/node]
11: 0x10036dfe0 v8::internal::PageEvacuationTask::RunInParallel(v8::internal::ItemParallelJob::Task::Runner) [/usr/local/bin/node]
12: 0x10034394c v8::internal::ItemParallelJob::Task::RunInternal() [/usr/local/bin/node]
13: 0x100343cd2 v8::internal::ItemParallelJob::Run() [/usr/local/bin/node]
14: 0x10035c36d void v8::internal::MarkCompactCollectorBase::CreateAndExecuteEvacuationTasks<v8::internal::FullEvacuator, v8::internal::MarkCompactCollector>(v8::internal::MarkCompactCollector*, v8::internal::ItemParallelJob*, v8::internal::MigrationObserver*, long) [/usr/local/bin/node]
15: 0x10035c14a v8::internal::MarkCompactCollector::EvacuatePagesInParallel() [/usr/local/bin/node]
16: 0x10034fb9a v8::internal::MarkCompactCollector::Evacuate() [/usr/local/bin/node]
17: 0x10034da42 v8::internal::MarkCompactCollector::CollectGarbage() [/usr/local/bin/node]
18: 0x100331b01 v8::internal::Heap::MarkCompact() [/usr/local/bin/node]
19: 0x10032fdad v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/usr/local/bin/node]
20: 0x10032e673 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/bin/node]
21: 0x10038b580 v8::internal::ScavengeJob::Task::RunInternal() [/usr/local/bin/node]
22: 0x10010b08a node::PerIsolatePlatformData::RunForegroundTask(std::__1::unique_ptr<v8::Task, std::__1::default_delete<v8::Task> >) [/usr/local/bin/node]
23: 0x10010a12b node::PerIsolatePlatformData::FlushForegroundTasksInternal() [/usr/local/bin/node]
24: 0x1007bddfa uv__async_io [/usr/local/bin/node]
25: 0x1007cdfc4 uv__io_poll [/usr/local/bin/node]
26: 0x1007be232 uv_run [/usr/local/bin/node]
27: 0x1000065ec node::SpinEventLoop(node::Environment*) [/usr/local/bin/node]
28: 0x1000e7326 node::NodeMainInstance::Run(node::EnvSerializeInfo const*) [/usr/local/bin/node]
29: 0x100085b8c node::Start(int, char**) [/usr/local/bin/node]
30: 0x7fff20467621 start [/usr/lib/system/libdyld.dylib]
31: 0x4
/usr/local/bin/zenbot: line 2: 78961 Abort trap: 6           env node zenbot.js $@

[acidtib]

I was able to pass this error by running zenbot with this env NODE_OPTIONS=--max-old-space-size=8192 do some test and bump the space size to fit your needs.

example:
NODE_ENV=production NODE_OPTIONS=--max-old-space-size=8192 zenbot trade --paper