Scheduled post #848

harshilsharma63 · 2024-11-14T06:29:09Z

Summary

Adding scheduled posts support to load test tool.

Ticket Link

Fixes https://mattermost.atlassian.net/browse/MM-61486

agnivade

It looks like some debug fmt.Println statements are still left over from debugging. Let's clean them up.

agarciamontoro

Thank you so much for all your work on this, great job! I left some comments below, and I also agree with Agniva: we should remove all the fmt.Println lines.

loadtest/control/simulcontroller/controller.go

agarciamontoro · 2024-11-18T13:29:16Z

loadtest/control/simulcontroller/scheduled_posts.go

+	if rand.Float64() < 0.02 {
+		if err := control.AttachFilesToDraft(u, &scheduledPost.Draft); err != nil {
+			fmt.Println("createScheduledPost: AttachFilesToDraft error", err)
+			return control.UserActionResponse{Err: control.NewUserError(err)}
+		}
+	}


We should probably move this 0.02 ratio to a global constant, we're using it in many places now.

Only in 5 places all over excluding tests. I don't think a constant for 2% makes sense. Its doesn't have any special meaning.

Well, I didn't mean the number 0.02 specifically, but a constant for when to attach files to a post, for which we use 2%. We use that exact meaning elsewhere (probably less than 5 places, but more than 1, I'd say)

loadtest/control/simulcontroller/scheduled_posts.go

loadtest/store/memstore/random.go

loadtest/user/userentity/scheduled_posts.go

loadtest/control/simulcontroller/scheduled_posts.go

loadtest/utils.go

agnivade · 2024-11-20T03:51:23Z

loadtest/store/memstore/store.go

@@ -52,6 +52,7 @@ type MemStore struct {
 	featureFlags        map[string]bool
 	report              *model.PerformanceReport
 	channelBookmarks    map[string]*model.ChannelBookmarkWithFileInfo
+	scheduledPosts      map[string]map[string]*model.ScheduledPost


Also, please update the (s *MemStore) Clear() method to clear this field as well.

agnivade · 2024-11-20T03:52:41Z

loadtest/store/memstore/random.go

+	return selectedPost, nil
+}
+
+func (s *MemStore) DeleteScheduledPost(scheduledPostID string) {


The delete and update methods should not be in random.go. Let's move this to memstore/store.go.

harshilsharma63 · 2024-12-02T05:10:41Z

@agnivade @agarciamontoro can you please re-review this? I've made the fixes.

agnivade

I will leave the store logic to Alejandro.

agnivade · 2024-12-02T05:26:45Z

loadtest/store/store.go

@@ -200,6 +204,10 @@ type MutableUserStore interface {
 	// SetDrafts stores the given drafts.
 	SetDrafts(teamId string, drafts []*model.Draft) error

+	// scheduled posts
+	SetScheduledPost(teamId string, scheduledPost *model.ScheduledPost) error
+	GetRandomScheduledPost() (*model.ScheduledPost, error)


MutableUserStore embeds UserStore. So there is no need to repeat GetRandomScheduledPost here. UserStore should have GetRandomScheduledPost. And MutableUserStore should have the Set/Update/Delete methods.

agnivade · 2024-12-02T05:28:06Z

loadtest/user/userentity/scheduled_posts.go

+	for teamIdInResponse := range scheduledPostsByTeam {
+		for _, scheduledPost := range scheduledPostsByTeam[teamIdInResponse] {


You can just do for _, v := range scheduledPostsByTeam { instead of 2 loops.

I'd still need two loops. scheduledPostsByTeam is a map[string][]*model.ScheduledPosts, and I need to iterate the individual scheduled posts from that array. I could use for _, v := range scheduledPostsByTeam but then I'd need another loop to iterate items from v.

Ah right. Indeed.

I'd prefer using the value from range as well, that way you don't need to access it later. Quite a nit, but I had to stop and try to understand why we weren't doing that, so I guess it's better to just stick to the convention.

loadtest/utils.go

agnivade

Looking good!

agarciamontoro

Thanks, @harshilsharma63, and sorry for the delay in the review! Some more comments below.

loadtest/control/simulcontroller/scheduled_posts.go

loadtest/store/memstore/store.go

agarciamontoro · 2024-12-10T15:04:56Z

loadtest/user/userentity/scheduled_posts.go

+	err = ue.store.SetScheduledPost(teamId, updatedScheduledPost)
+	if err != nil {
+		return err
+	}
+
+	ue.Store().UpdateScheduledPost(teamId, updatedScheduledPost)


Do we need to set it and then update it? Also, we should check the error from UpdateScheduledPost

Just update should be fine, fixed this.

agarciamontoro · 2024-12-10T15:05:26Z

loadtest/store/memstore/store.go

+func (s *MemStore) DeleteScheduledPost(scheduledPostID string) {
+	s.lock.Lock()
+	defer s.lock.Unlock()
+
+	for teamId := range s.scheduledPosts {
+		if _, ok := s.scheduledPosts[teamId][scheduledPostID]; ok {
+			delete(s.scheduledPosts[teamId], scheduledPostID)
+			break
+		}
+	}
+}
+


Should we error if the post doesn't exist?

It doesn't really matter.

agarciamontoro · 2024-12-10T15:06:03Z

loadtest/user/userentity/scheduled_posts.go

+	ue.Store().DeleteScheduledPost(scheduledPostId)
+	return nil


Right now DeleteScheduledPost doesn't return an error, but maybe it should (I left a comment for that). If we decide that's a good decision, then we need to return its error.

I don't think its of any use returning error from delete method, it doesn't matter

agarciamontoro · 2024-12-10T15:07:31Z

loadtest/user/userentity/scheduled_posts.go

+	for teamIdInResponse := range scheduledPostsByTeam {
+		for _, scheduledPost := range scheduledPostsByTeam[teamIdInResponse] {


I'd prefer using the value from range as well, that way you don't need to access it later. Quite a nit, but I had to stop and try to understand why we weren't doing that, so I guess it's better to just stick to the convention.

loadtest/utils.go

agarciamontoro

Thanks, @harshilsharma63! I've left a comment in the RandomFutureTime function, and I have an additional ask: I do think we should error out in MemStore's DeleteScheduledPost and UpdateScheduledPost if the post is not found, and log an error if that's the case. It shouldn't happen and may point to other problems in the load-test.

agarciamontoro · 2024-12-19T15:12:56Z

loadtest/utils.go

+// RandomFutureTime returns a random Unix timestamp, in milliseconds, in the interval
+// [now+deltaStart, now+maxUntil]


The comment is actually correct, thanks! But it surfaced a bug in this function: if maxUntil is smaller than deltaStart (let's say, you call RandomFutureTime(10*time.Second, 9*time.Second), then rand.Int63n panics because it gets a negative number). We should fix the function so that the interval is actually [now+deltaStart, now+deltaStart+maxUntil], and add unit testing for this function.

agnivade · 2024-12-23T07:48:55Z

Both Alejandro and Claudio are out. I will take another quick look and ask for a stamp approval.

agnivade · 2024-12-23T09:32:27Z

I gave it a test run and found some minor issues. We should be good to go after that.

1. Incorrect error/info logging when there are no scheduled posts.

Log line is:

error [2024-12-23 14:51:17.856 +05:30] control/simulcontroller.(*SimulController).sendScheduledPostNow loadtest/control/simulcontroller/scheduled_posts.go:135 no scheduled posts available caller="loadtest/loadte
st.go:60" controller_id=4 user_id=anoxyyicofbozjrig7og95i6py
...
error [2024-12-23 14:51:34.667 +05:30] control/simulcontroller.(*SimulController).deleteScheduledPost loadtest/control/simulcontroller/scheduled_posts.go:113 no scheduled posts available caller="loadtest/loadtes
t.go:60" controller_id=10 user_id=bpjjn1f67jg8xnya68m41ki6dc

Note that the log level is error, but it's just a case of scheduled posts not being available. And we are correctly doing it here:

scheduledPost, err := u.Store().GetRandomScheduledPost()
if err != nil {
return control.UserActionResponse{Err: control.NewUserError(err)}
}
if scheduledPost == nil {
return control.UserActionResponse{Info: "no scheduled posts found"}
}

But the issue is that GetRandomScheduledPost() already returns an error if no scheduled posts are found.

The right way is to return an error variable and check for it. See for example RandomTeam:

	if len(teams) == 0 {
		return model.Team{}, ErrTeamStoreEmpty
	}

and then we check for the error at call site:

team, err := u.Store().RandomTeam(store.SelectMemberOf | store.SelectNotCurrent)
if errors.Is(err, memstore.ErrTeamStoreEmpty) {
return control.UserActionResponse{Info: "no other team to switch to"}
} else if err != nil {
return control.UserActionResponse{Err: control.NewUserError(err)}
}

We should apply the same logic while calling GetRandomScheduledPost as well.

2. Invalid scheduled at time:

error [2024-12-23 14:52:18.420 +05:30] control/simulcontroller.(*SimulController).createScheduledPost loadtest/control/simulcontroller/scheduled_posts.go:63 Invalid scheduled at time., id=m9txr5p37bf63kg4xwq7upjxcy caller="loadtest/loadtest.go:60" controller_id=9 user_id=b1ysdcjc5prodfef1386jdjawy

This is what I see in the server logs:

debug [2024-12-23 15:01:16.543 +05:30] Invalid scheduled at time.                    caller="web/context.go:120" path=/api/v4/posts/schedule request_id=xzx84obdgjgdugma4setb8a1ye ip_addr=127.0.0.1 user_id=eh1k7cuhmtntfx8y9ziihatmkr method=POST err_where=ScheduledPost.IsValid http_code=400 error="ScheduledPost.IsValid: Invalid scheduled at time., id=qoafc3rp3tf5zjsqpcck9ew55w"

If we fix the above 2 issues, we should be good to go.

harshilsharma63 added 11 commits October 28, 2024 15:40

Added create scheduled posts

22cdf28

Addded UpdateScheduledPost

4341570

DeleteScheduledPost

f51ece6

SendScheduledPost

28a8187

Updated loadTeam

18fd076

WIP

1e816df

WIP

a89e45f

WIP

532675b

Merge branch 'master' into scheduled_post

4f500e3

Cleanup

6394486

Comment cleanup

d37d54e

harshilsharma63 marked this pull request as ready for review November 14, 2024 07:01

harshilsharma63 requested review from agarciamontoro and streamer45 November 14, 2024 07:01

agnivade reviewed Nov 16, 2024

View reviewed changes

agarciamontoro requested changes Nov 18, 2024

View reviewed changes

agnivade requested changes Nov 20, 2024

View reviewed changes

harshilsharma63 added 3 commits November 21, 2024 14:08

Merge branch 'master' into scheduled_post

b862d79

Merge branch 'master' into scheduled_post

3e0bf55

review fixes

2e439ab

harshilsharma63 requested review from agnivade and agarciamontoro November 27, 2024 04:44

agnivade reviewed Dec 2, 2024

View reviewed changes

harshilsharma63 added 3 commits December 9, 2024 06:34

Merge branch 'master' into scheduled_post

a48e09a

Cleanup

1e2606e

Cleanup

8096d4a

harshilsharma63 requested a review from agnivade December 9, 2024 02:57

agnivade approved these changes Dec 9, 2024

View reviewed changes

agarciamontoro requested changes Dec 10, 2024

View reviewed changes

harshilsharma63 added 2 commits December 19, 2024 09:08

Merge branch 'master' into scheduled_post

054353f

review Fixes

1c9a250

harshilsharma63 requested a review from agarciamontoro December 19, 2024 04:22

agarciamontoro requested changes Dec 19, 2024

View reviewed changes

harshilsharma63 added 2 commits December 23, 2024 11:17

Merge branch 'master' into scheduled_post

f249da9

Updated RandomFutureTime

fe0df4b

harshilsharma63 requested a review from agarciamontoro December 23, 2024 06:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Scheduled post #848

Scheduled post #848

harshilsharma63 commented Nov 14, 2024

agnivade left a comment

agarciamontoro left a comment

agarciamontoro Nov 18, 2024

harshilsharma63 Nov 21, 2024

agarciamontoro Dec 2, 2024

harshilsharma63 Dec 9, 2024

agnivade Nov 20, 2024

harshilsharma63 Nov 27, 2024

agnivade Nov 20, 2024

harshilsharma63 Nov 27, 2024

harshilsharma63 commented Dec 2, 2024

agnivade left a comment

agnivade Dec 2, 2024

harshilsharma63 Dec 9, 2024

agnivade Dec 2, 2024

harshilsharma63 Dec 9, 2024

agnivade Dec 9, 2024

agarciamontoro Dec 10, 2024

harshilsharma63 Dec 19, 2024

agnivade left a comment

agarciamontoro left a comment

agarciamontoro Dec 10, 2024

harshilsharma63 Dec 19, 2024

agarciamontoro Dec 10, 2024

harshilsharma63 Dec 19, 2024

agarciamontoro Dec 10, 2024

harshilsharma63 Dec 19, 2024

agarciamontoro Dec 10, 2024

agarciamontoro left a comment

agarciamontoro Dec 19, 2024

harshilsharma63 Dec 23, 2024

agnivade commented Dec 23, 2024

agnivade commented Dec 23, 2024

		for teamIdInResponse := range scheduledPostsByTeam {
		for _, scheduledPost := range scheduledPostsByTeam[teamIdInResponse] {

		// RandomFutureTime returns a random Unix timestamp, in milliseconds, in the interval
		// [now+deltaStart, now+maxUntil]

Scheduled post #848

Are you sure you want to change the base?

Scheduled post #848

Conversation

harshilsharma63 commented Nov 14, 2024

Summary

Ticket Link

agnivade left a comment

Choose a reason for hiding this comment

agarciamontoro left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

harshilsharma63 commented Dec 2, 2024

agnivade left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

agnivade left a comment

Choose a reason for hiding this comment

agarciamontoro left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

agarciamontoro left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

agnivade commented Dec 23, 2024

agnivade commented Dec 23, 2024