Recommendations for attack-based instruments?

[bluenote10]

So far the most impressive examples of ddsp I've seen/heard were using instruments that don't have sharp attacks, like wind or bowed instruments. In these cases physical energy is transferred into the instrument more constantly, and the related noise profiles are also less transient, which conceptually plays well with the ddsp model.

I'm wondering if there are any recommendations how to get better results out of ddsp for instruments that have a sharp attack like strummed/plucked instruments (guitar, piano, ...). I'm currently trying to train a basic acoustic guitar model. My training data is based on this ~14 min recording of me playing some random monophonic notes.

Regarding training I was using the default models/solo_instrument.gin using the default command:

!ddsp_run \
  --mode=train \
  --alsologtostderr \
  --save_dir="$SAVE_DIR" \
  --gin_file=models/solo_instrument.gin \
  --gin_file=datasets/tfrecord.gin \
  --gin_param="TFRecordProvider.file_pattern='$TRAIN_TFRECORD_FILEPATTERN'" \
  --gin_param="batch_size=16" \
  --gin_param="train_util.train.num_steps=30000" \
  --gin_param="train_util.train.steps_per_save=50" \
  --gin_param="trainers.Trainer.checkpoints_to_keep=10"

The training seemed to have converged reasonably well, ending up in the 4.5-5.0 loss range after ~22k steps.

Now I'm trying to resynthesize some examples based on the original training data. The results are not too bad, but still sound relatively artificial. I've noticed two main issues in particular:

• For some reason the resynthesized examples sometimes have way too much reverb, which seems to be a major contributor to making them sound artificial. The first example below demonstrates that.
• Sometimes the attacks (plucks) themselves are washed-out, which also removes a lot of the guitar characteristics.

Examples:

• Example 1 (orig)
• Example 1 (synth)
• Example 2 (orig)
• Example 2 (synth)

Has anyone done some experiments with attack-based instruments already? I'd be curious if some parameter tweaking can help here. Perhaps a dedicated .gin file for attack-based instruments would make sense?

I'd assume that at least the first issue could be mitigated by reducing the influence of the reverb, but I'm not quite sure how or why it is happening in the first place. The second issues might be more tricky, because the sharp note onset probably is tougher to handle in terms of estimation of loudness / f0 / f0_confidence. Perhaps making the model more sensitive (less f0_confidence filtering) or so could help?