Improve SOC documentation

[agazso]

When using the SOCReader or SOCWriter interfaces they expect a parameter called identifier with type Identifier (which is just a type alias for Bytes<32>). However on the public interface there are no constructor functions for this type, so it is not really clear how to make such an identifier.

Real world example:
https://github.com/significance/swapchat-engine/blob/b1554f82ed0fc1ba3d7514bd2c27ba136187f88f/src/swarm.ts#L38

It would be good if there were some example functions on the public interface, maybe as utility function. One candidate could be the makeSequentialFeedIdentifier function.

[AuHau]

So I am not really sure how to go about this.

The Identifier in SOC context as described in Book of Swarm can be whatever unless it is 32 bytes. There is no specification on what does it mean only that it can be written to it only once. Because of that, I am not sure what should the "identifier constructor" actually do. We do not have any internal "factory for identifiers" that we could expose.

We could have for example factory which could take in a string, hash it and use it as an identifier. That is possible, but I am not sure if it is a good idea. From my point of view, SOCs are very low level and most users should not be using them unless they need them for some special use-case and they know what they are doing (and then they should know how to construct identifier). The caveat here is that the user needs to make sure he is not re-publishing the chunk with the same identifier as "the behavior of the network is unpredictable". And I am quite sure if we would provide the factory that I have mentioned people would use it in Feed's topic sort of style which would lead to unpredictable behavior.

That said I agree that there is a space for improvement on the API level to better communicate what Identifier is, which to my understanding would solve the problem that @significance had in the swapchat.

[significance]

yep i think that's cool if i did it the way it is expected do it?

[agazso]

The Identifier in SOC context as described in Book of Swarm can be whatever unless it is 32 bytes. There is no specification on what does it mean only that it can be written to it only once. Because of that, I am not sure what should the "identifier constructor" actually do. We do not have any internal "factory for identifiers" that we could expose.

There is one use-case that is already supported in bee-js and there were feature requests about it and that is the feeds use-case. Feeds are built on top of SOCs and use a certain convention (also described in the Book of Swarm) how to construct the identifier. People want to access the individual updates or get the address of them. That's why I proposed to expose the makeSequentialFeedIdentifier function as a helper function on the public interface (maybe with a better name).

Once that is exposed, then the SOCReader interface can be used to access any feed updates with a given index. I can also imagine extending the SOCReader interface with a makeAddress function that expects an identifier and returns the ChunkAddress for the given chunk. With that you would also be able to verify if a feed update with a given index is retrievable from the network (combined with the isReferenceRetrievable function).

I agree that SOCs are low level, however if we expose them on the public interface (and we indeed do already) then we should be aiming for making them easy to use as well. It's already possible to upload to the same identifier twice, but in my opinion if we make the SOC interface hard to use then it's more likely that people will misuse it and it may lead to errors that are hard to understand as well. I have the feeling that a lot of people are mystified about feeds because they don't understand the underlying concept that are SOCs.

Also I think we may be opinionated when defining helper functions that make it easier to use with opaque byte arrays. For example there is already a makeFeedTopic function that takes a string and returns a Topic type by simply hashing the input string.

[AuHau]

So I see two parts in your answer:

1. Improving SOC component usability
2. Improving Feeds capabilities

I agree with both of those, but I am not so on board with the way how to achieve these.

Regarding improving SOC usability, as I have already suggested improving the API and documentation for SOC interfaces should be definitely tackled, but again there should be a big warning around SOC functionality that it is low level primitive that is not directly meant to be used for high-level use cases and that the user needs to understand the consequences when using it (which also requires proper documentation of course).

Regarding the Feeds support. I agree that these use cases are needed but instead of forcing the users to use SOC for these functionalities, I would then create proper support for these in the realm of Feeds itself. From my understanding it would be beneficial to have support for these use cases:

• get the state of feed for the given index (you can get already the current feed index when you fetch Feed with FeedReader)
• have isFeedRetrievable(index='latest') function

Anything more?

[AuHau]

I have created issues for the the use cases and have rephrased this issue to tackle the documentation and API changes for the SOC.

The new issues: #634 #633