DAOS-14698 crtl: Create container with attributes #15300

knard38 · 2024-10-11T21:23:43Z

Description

Add new option attr to daos cont create allowing to define container user attributes.

Before requesting gatekeeper:

Two review approvals and any prior change requests have been resolved.
Testing is complete and all tests passed or there is a reason documented in the PR why it should be force landed and forced-landing tag is set.
Features: (or Test-tag*) commit pragma was used or there is a reason documented that there are no appropriate tags for this PR.
Commit messages follows the guidelines outlined here.
Any tests skipped by the ticket being addressed have been run and passed in the PR.

Gatekeeper:

github-actions · 2024-10-11T21:24:02Z

Ticket title is 'Creating daos container with dfuse cache settings.'
Status is 'In Review'
Labels: 'triaged,usability'
Errors are Unknown component
https://daosio.atlassian.net/browse/DAOS-14698

Add new option set-attr to daos cont create allowing to define container user attributes. Features: control dfuse Required-githooks: true Signed-off-by: Cedric Koch-Hofer <[email protected]>

daosbuild1 · 2024-10-18T16:00:49Z

Test stage NLT on EL 8.8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-15300/4/testReport/

Update and add new functional tests. Features: control dfuse container Required-githooks: true Signed-off-by: Cedric Koch-Hofer <[email protected]>

src/tests/ftest/dfuse/container_attrs.py

daosbuild1 · 2024-10-21T10:49:27Z

Test stage NLT on EL 8.8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-15300/6/testReport/

Integrate reviewers comments: - Move check_attrs() to private Features: control dfuse container Required-githooks: true Signed-off-by: Cedric Koch-Hofer <[email protected]>

daosbuild1 · 2024-10-21T12:54:10Z

Test stage NLT on EL 8.8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-15300/7/testReport/

mchaarawi · 2024-10-21T14:12:57Z

src/include/daos_uns.h

+duns_create_path_attr(daos_handle_t poh, const char *path, int count, char const *const names[],
+		      void const *const values[], size_t const sizes[], struct duns_attr_t *attrp);


im not a fan of this new API. it might be useful to have something for the command line.
but for the API, it's not a big deal to do
daos_cont_create()
daos_cont_set_attr()
I would suggest to remove it.

Not sure to fully understand your comments.
Are you suggesting to remove duns_create_path_attr() and duns_create_path() functions or just the first one.
If we are just removing duns_create_path_attr(), then we will have two solutions from my understanding:

Change the dfuse code to poll the attributes of its mounted containers.

split the function duns_create_path_attr() to not create and bind the container in the same function. By this way, we will be able to use the function daos_cont_set_attr() between the two function calls.

no you cannot remove duns_create_path()!
my question is why you are adding this new API (duns_crreate_path_attr()) ?
what is the challenge of doing:
duns_create_path()
daos_cont_set_attr()
?

From my understanding, doing this will not work as the attributes are cached inside DFuse after the binding.
Thus, when we will get back from duns_create_path() , the call to daos_cont_set_attr() will not be taken into account by DFuse.
@ashleypittman could you confirm this.

I've taken a closer look and think you're both right, having a new API doesn't seem like the right solution here, however I hadn't realised it was the same C function that both created the container and inserted it into the POSIX namespace.

The best path might be:
daos_cont_create()
daos_cont_set_attr()
duns_link_cont()

I was not aware of the duns_link_cont() function, which for sure is a better solution.
Thanks @ashleypittman for the hints and @mchaarawi for the relevant remark :-)

Remove useless new function duns_create_path_attr()

Split sub container creation

I was not aware of the duns_link_cont() function, which for sure is a better solution. Thanks @ashleypittman for the hints and @mchaarawi for the relevant remark :-)

Remove useless new function duns_create_path_attr()

Split sub container creation

Fixed with commit 819b856

kjacque

From the Go side, the approach looks fine to me. Will wait for the other requested changes to come through before approving.

src/control/cmd/daos/container.go

daosbuild1 · 2024-10-21T22:41:57Z

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-15300/7/execution/node/1217/log

…/daos-14698 Required-githooks: true

daosbuild1 · 2024-11-23T10:16:24Z

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-15300/35/execution/node/1210/log

knard38 · 2024-11-25T13:17:23Z

The failing test pool/query_attribute.py is a regression recently introduced on master.
I am going to open JIRA ticket to solved

…/daos-14698 Restart CI with excluding unealthy test Features: control dfuse container daos_cmd Skip-list: QueryAttributeTest,test_query_attr:16829 Required-githooks: true

daosbuild1 · 2024-11-26T09:54:20Z

Test stage NLT on EL 8.8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-15300/36/testReport/

daosbuild1 · 2024-11-26T16:58:34Z

Test stage NLT on EL 8.8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-15300/37/testReport/

…/daos-14698 Required-githooks: true

knard38 · 2024-11-28T17:13:09Z

Increase maximal log size of NLT size as last test failed with the following message:

Max log size exceeded, 1.7GiB > 1.7GiB

knard38 · 2024-11-28T17:16:39Z

Force push was just for updating the commit message to skip known functional test failure.

Fix NLT max log size. Features: control dfuse container daos_cmd Required-githooks: true Signed-off-by: Cedric Koch-Hofer <[email protected]>

knard38 · 2024-11-29T07:48:18Z

Force push again as the skip-test macros is not working as expected.

daosbuild1 · 2024-11-29T21:05:09Z

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-15300/40/execution/node/1210/log

…/daos-14698 Restart CI with excluding unealthy test Features: control dfuse container daos_cmd Skip-list: QueryAttributeTest,test_query_attr:16829 DfuseMUPerms,test_dfuse_mu_perms:16695 Required-githooks: true

…/daos-14698 Restart CI without excluding unealthy test Features: control dfuse container daos_cmd

daosbuild1 · 2024-12-06T21:04:17Z

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-15300/45/execution/node/1211/log

…/daos-14698 Allow-unstable-test: true Features: control dfuse container daos_cmd Required-githooks: true Signed-off-by: Cedric Koch-Hofer <[email protected]>

daosbuild1 · 2024-12-10T07:21:18Z

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-15300/48/execution/node/1210/log

knard38 · 2024-12-11T07:05:00Z

Hello reviewers,

Hopefully, I should have integrated all your comments :-)

@mchaarawi could you have a look at the C code and NLT test to check if the modifications are OK for you.
@mjmac and/or @kjacque could you have a look at the GO code to check if the modifications are OK for you.
@daltonbohning and/or @phender could you have a look at the functional test parts to check if the modifications are OK for you.

The two errors of the functional tests are known issues:

dfuse/mu_perms.py: related to ticket DAOS-16695
pool/query_attribute.py: related to ticket DAOS-16829

Thanks in advance,
Cedric.

knard38 · 2024-12-12T08:37:16Z

@daos-stack/daos-gatekeeper please could you land this PR with the following message:

title: DAOS-14698 crtl: Create container with attributes
body:

Add new option attr to daos cont create allowing to define container user attributes.

daltonbohning · 2024-12-12T20:33:35Z

Looks like no more changes to GO code after go-owners last reviewed, so not requiring re-review.

knard38 force-pushed the ckochhof/dev/master/daos-14698 branch 2 times, most recently from 3a1167a to 7d8cdd0 Compare October 16, 2024 09:58

ashleypittman mentioned this pull request Oct 17, 2024

DAOS-16568 tests: Better cmdline control of NLT tests #15141

Merged

18 tasks

DAOS-14698 crtl: Create container with attributes

5fcd159

Add new option set-attr to daos cont create allowing to define container user attributes. Features: control dfuse Required-githooks: true Signed-off-by: Cedric Koch-Hofer <[email protected]>

knard38 force-pushed the ckochhof/dev/master/daos-14698 branch from 7d8cdd0 to 77f33b2 Compare October 18, 2024 15:13

knard38 force-pushed the ckochhof/dev/master/daos-14698 branch from 77f33b2 to 8421816 Compare October 21, 2024 10:01

DAOS-14698 crtl: Create container with attributes

315f440

Update and add new functional tests. Features: control dfuse container Required-githooks: true Signed-off-by: Cedric Koch-Hofer <[email protected]>

knard38 force-pushed the ckochhof/dev/master/daos-14698 branch from 8421816 to 315f440 Compare October 21, 2024 10:05

knard38 marked this pull request as ready for review October 21, 2024 10:27

knard38 requested review from a team as code owners October 21, 2024 10:27

knard38 requested review from ashleypittman, daltonbohning, mjmac and mchaarawi October 21, 2024 10:28

ashleypittman reviewed Oct 21, 2024

View reviewed changes

src/tests/ftest/dfuse/container_attrs.py Outdated Show resolved Hide resolved

src/tests/ftest/dfuse/container_attrs.py Show resolved Hide resolved

knard38 requested a review from ashleypittman October 21, 2024 12:10

DAOS-14698 crtl: Create container with attributes

865f2e5

Integrate reviewers comments: - Move check_attrs() to private Features: control dfuse container Required-githooks: true Signed-off-by: Cedric Koch-Hofer <[email protected]>

mchaarawi reviewed Oct 21, 2024

View reviewed changes

kjacque reviewed Oct 21, 2024

View reviewed changes

src/control/cmd/daos/container.go Show resolved Hide resolved

Merge remote-tracking branch 'origin/master' into ckochhof/dev/master…

4dd51bd

…/daos-14698 Required-githooks: true

Merge remote-tracking branch 'origin/master' into ckochhof/dev/master…

34614ba

…/daos-14698 Restart CI with excluding unealthy test Features: control dfuse container daos_cmd Skip-list: QueryAttributeTest,test_query_attr:16829 Required-githooks: true

Merge remote-tracking branch 'origin/master' into ckochhof/dev/master…

0afbeac

…/daos-14698 Required-githooks: true

knard38 force-pushed the ckochhof/dev/master/daos-14698 branch from f6e3058 to 1078650 Compare November 28, 2024 17:15

DAOS-14698 crtl: Create container with attributes

d5a474a

Fix NLT max log size. Features: control dfuse container daos_cmd Required-githooks: true Signed-off-by: Cedric Koch-Hofer <[email protected]>

knard38 force-pushed the ckochhof/dev/master/daos-14698 branch from 1078650 to d5a474a Compare November 29, 2024 07:47

kanard38 added 3 commits December 4, 2024 07:31

Merge remote-tracking branch 'origin/master' into ckochhof/dev/master…

5264e23

…/daos-14698 Restart CI with excluding unealthy test Features: control dfuse container daos_cmd Skip-list: QueryAttributeTest,test_query_attr:16829 DfuseMUPerms,test_dfuse_mu_perms:16695 Required-githooks: true

Merge remote-tracking branch 'origin/master' into ckochhof/dev/master…

a8d3da6

…/daos-14698 Restart CI without excluding unealthy test Features: control dfuse container daos_cmd

Merge remote-tracking branch 'origin/master' into ckochhof/dev/master…

61c000a

…/daos-14698 Restart CI without excluding unealthy test Features: control dfuse container daos_cmd

Merge remote-tracking branch 'origin/master' into ckochhof/dev/master…

a90bf29

…/daos-14698 Allow-unstable-test: true Features: control dfuse container daos_cmd Required-githooks: true Signed-off-by: Cedric Koch-Hofer <[email protected]>

knard38 requested review from phender and kjacque December 11, 2024 06:57

mchaarawi approved these changes Dec 11, 2024

View reviewed changes

daltonbohning approved these changes Dec 11, 2024

View reviewed changes

knard38 requested review from a team and removed request for ashleypittman December 12, 2024 08:37

daltonbohning merged commit c931b5f into master Dec 12, 2024
72 of 80 checks passed

daltonbohning deleted the ckochhof/dev/master/daos-14698 branch December 12, 2024 20:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DAOS-14698 crtl: Create container with attributes #15300

DAOS-14698 crtl: Create container with attributes #15300

knard38 commented Oct 11, 2024 •

edited

Loading

github-actions bot commented Oct 11, 2024 •

edited

Loading

daosbuild1 commented Oct 18, 2024

daosbuild1 commented Oct 21, 2024

daosbuild1 commented Oct 21, 2024

mchaarawi Oct 21, 2024

knard38 Oct 21, 2024

mchaarawi Oct 21, 2024

knard38 Oct 21, 2024

ashleypittman Oct 21, 2024

knard38 Oct 21, 2024 •

edited

Loading

knard38 Oct 22, 2024

kjacque left a comment

daosbuild1 commented Oct 21, 2024

daosbuild1 commented Nov 23, 2024

knard38 commented Nov 25, 2024 •

edited

Loading

daosbuild1 commented Nov 26, 2024

daosbuild1 commented Nov 26, 2024

knard38 commented Nov 28, 2024

knard38 commented Nov 28, 2024

knard38 commented Nov 29, 2024

daosbuild1 commented Nov 29, 2024

daosbuild1 commented Dec 6, 2024

daosbuild1 commented Dec 10, 2024

knard38 commented Dec 11, 2024

knard38 commented Dec 12, 2024

daltonbohning commented Dec 12, 2024

		duns_create_path_attr(daos_handle_t poh, const char path, int count, char const const names[],
		void const const values[], size_t const sizes[], struct duns_attr_t attrp);

DAOS-14698 crtl: Create container with attributes #15300

DAOS-14698 crtl: Create container with attributes #15300

Conversation

knard38 commented Oct 11, 2024 • edited Loading

Description

Before requesting gatekeeper:

Gatekeeper:

github-actions bot commented Oct 11, 2024 • edited Loading

daosbuild1 commented Oct 18, 2024

daosbuild1 commented Oct 21, 2024

daosbuild1 commented Oct 21, 2024

mchaarawi Oct 21, 2024

Choose a reason for hiding this comment

knard38 Oct 21, 2024

Choose a reason for hiding this comment

mchaarawi Oct 21, 2024

Choose a reason for hiding this comment

knard38 Oct 21, 2024

Choose a reason for hiding this comment

ashleypittman Oct 21, 2024

Choose a reason for hiding this comment

knard38 Oct 21, 2024 • edited Loading

Choose a reason for hiding this comment

knard38 Oct 22, 2024

Choose a reason for hiding this comment

kjacque left a comment

Choose a reason for hiding this comment

daosbuild1 commented Oct 21, 2024

daosbuild1 commented Nov 23, 2024

knard38 commented Nov 25, 2024 • edited Loading

daosbuild1 commented Nov 26, 2024

daosbuild1 commented Nov 26, 2024

knard38 commented Nov 28, 2024

knard38 commented Nov 28, 2024

knard38 commented Nov 29, 2024

daosbuild1 commented Nov 29, 2024

daosbuild1 commented Dec 6, 2024

daosbuild1 commented Dec 10, 2024

knard38 commented Dec 11, 2024

knard38 commented Dec 12, 2024

daltonbohning commented Dec 12, 2024

knard38 commented Oct 11, 2024 •

edited

Loading

github-actions bot commented Oct 11, 2024 •

edited

Loading

knard38 Oct 21, 2024 •

edited

Loading

knard38 commented Nov 25, 2024 •

edited

Loading