Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: sequential feed #250

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft

feat: sequential feed #250

wants to merge 3 commits into from

Conversation

IgorShadurin
Copy link
Collaborator

@IgorShadurin IgorShadurin commented Jul 27, 2023

BREAKING CHANGE!

Implemented ability to use not only Epoch feed, but also Sequence feed.

Epoch vs. Sequence Feed types: speed comparison

The performance of the two types of feeds, Epoch and Sequence, in Bee was tested under two conditions: using a local Bee node and a remote FDS Bee node. The same script was used to test both feed types to ensure consistent conditions. The script performed the following tasks:

  1. Created a new FDP instance using either the Epoch or Sequence feed type.
  2. Generated two random identifiers and created two pods with them.
  3. Generated two random directories and created these directories in each pod.
  4. Uploaded a randomly generated file to each pod.
  5. Downloaded file from each pod.
  6. Read the root directory of each pod in recursive mode.

Here's a comparison of the execution times:

Local Bee Node Test:

  • Epoch: 10.6 seconds
  • Sequence: 12.6 seconds

Remote FDS Bee Node Test (https://bee-1.fairdatasociety.org/):

  • Epoch: 5 minutes 11 seconds (311 seconds)
  • Sequential: 2 minutes 36 seconds (156 seconds)

The Sequence feed took 18.9% longer than the Epoch feed when tested on the local Bee node. However, when tested on the remote FDS Bee node, the Sequence feed executed in just 50.2% of the time it took the Epoch feed, achieving a 49.8% improvement in execution time.

Despite the speed increase, it's worth clarifying with the Bee team about the possibility of parallel data uploading in Sequential feed for use in FairOS. If they've fixed this, it will work not only in fdp-storage but also in FairOS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant