Platform event limits

[RaviVaranasi]

Hello,

Thanks for developing this package. We had an incident in production due to the daily platform events delivered limit. Is there any document/technique to overcome this without hitting the platform events? To provide context, we are making API callouts to DataDog to store the logs, so we don't need the Log Entry objects on the SF side. We are purging the log entries every three days.

We want to configure this to a mechanism that scales with our needs. Are there any tradeoffs or performance benchmarks with the different Save Method configuration, i.e., Event_Bus vs Queueable?

Thanks.

[jongpie]

Hi @RaviVaranasi - I need to better document how the different save methods work, the current documentation for the whole project is not as thorough as it should be. I'm working on improving the docs, but offhand, I think you have a few options on how to mitigate hitting your prod limit of daily platform events.

Used Different Save Methods

• EVENT_BUS is the most reliable option for storing logging data - platform events will still be published even if an exception occurs in Apex/Flow, so it's the best way to handle error logging
• For code paths where you don't anticipate any exceptions being thrown, you could consider using the save method SYNCHRONOUS_DML. This uses normal DML statements to create records in Log__c, LogEntry__c, etc,
  • Pro: It completely circumvents the EventBus, so it would not count towards you daily platform event publishing allocations.
  • Con: If any exceptions do occur in a transaction, this save method will fail to save the data (the platform automatically rolls back the data when an exception occurs), so you will lose logging data.
  • Con: the DML statements will synchronously execute Nebula Logger's triggers for Log__c, LogEntry__c, which will count towards your own code's transactional limits in Apex, like the limit on DML statements, CPU time, etc.
• QUEUEABLE starts an async queueable job, but within the queueable job, it's still using the EventBus to publish platform events. This could be possibly enhanced in the future to skip using the EventBus and instead use synchronous DML, but at least in current versions of Nebula Logger, this save method unfortunately won't help you reduce your platform event usage.
  • Given that your goal is ultimately to send the logging data to DataDog (and not to store it in Salesforce), I think exploring options for enhancing the queueable job would probably be the best area in Nebula Logger to enhance to help with your situation. I'll look into this more when I have time to do so.

Use Logging Levels To Manage How Much Logging Data Is Generated Per Profile Or User

• One other possible way to reduce your platform event usage is to make sure you're using logging methods that are appropriate for the data being logged, combined with customizing Nebula Logger's behavior using the custom settings object LoggerSettings__c.
  • Some orgs will use Logger.debug() for most statements, but it's best to come with a strategy (if you haven't already) to decide when to use each of the logging-level methods Logger.error(), Logger.warn(), Logger.info(), Logger.debug(), Logger.fine(), Logger.finer(), and Logger.finest().
  • You can then use the custom settings LoggerSettings__c in your production org to customize Nebula Logger org-wide, profile-level, and user-level for what logging data is saved (or not saved). For production orgs, I typically recommend setting LoggerSettings__c.LoggingLevel__c to INFO

Use Logger Scenarios To Manage How Much Logging Data Is Generated Per Scenario/Code Module

• If you have a large codebase with a lot of metadata, you can have different parts of your codebase use different options for Nebula Logger, using the custom metadata type LoggerScenarioRule__mdt.
• Scenario rules let you override LoggerSettings__c at runtime for certain parts of your codebase (which you control/indicate in Apex, using Logger.setScenario(String)). This would let you store granular logging data in one part of your codebase (e.g., store logging level FINEST data), while using completely different options elsewhere in your codebase.

[jongpie]

@RaviVaranasi a follow up question for you - how are you sending the data from Salesforce to DataDog? Are you using a queueable, batchable, etc to push the data to DataDog?