Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Q: What characters are legal / recommended for test name in YAML? #144

Open
ghost opened this issue Jun 9, 2016 · 5 comments · Fixed by #157
Open

Q: What characters are legal / recommended for test name in YAML? #144

ghost opened this issue Jun 9, 2016 · 5 comments · Fixed by #157

Comments

@ghost
Copy link

ghost commented Jun 9, 2016

I suspect plain alphanum + spaces is recommended, but I might sometimes want to do hyphen or parens. So, just being thorough.

Thanks!

@cdent
Copy link
Owner

cdent commented Jun 9, 2016

I was talking with someone about the first issue you raised and I thought: This would be a great time to start a FAQ. So keep em coming, these will all make fine faq starters.

In this particular case the honest answer is "dunno". I've not encountered a situation where I've tried to name a test and it hasn't worked, but that is probably because I'm biased in what I'm naming tests.

The full test name is the combination of what's in test_loader_name, the name of the yaml file and the the name attribute in each test (with spaces turned into underscores), and is, strictly speaking, a Python class.

To enable concurrent process grouping regex in .testr.conf (or other ways of doing it for other multi-process runners) to work out okay it is a good idea to not use _ in the name of the yaml file.

I've not proved out the limits of the test names. Probably something hypothesis would help with. Some casual testing in gabbi-run suggest that hyphen and parens are okay.

Do you think there should be defined restrictions? My attitude so far is that if you do something wrong, as long as it tells you very early in the process and with a reasonable error message, then it's okay.

@ghost
Copy link
Author

ghost commented Jun 9, 2016

I'm good with there not being defined restrictions. I think a FAQ would be great. You could use the Github Wiki for that perhaps. Thanks for looking into this!

p.s. rather than opening another issue immediately, another quick question: Is it possible to source integers from environment variables in the YAML? I ask because I just tried to source poll delay and count from environ, but test failed complaining about strings vs. integers. I tried unquoting the value during shell export, but no dice. Worth opening a new issue?

@cdent
Copy link
Owner

cdent commented Jun 9, 2016

made #145 to keep a reminder active for making a FAQ

The issue with strings and ints is because environment variables are always strings, see this block of code. In that block it would be possible to turn things that look like ints into its.

However, poll and count, don't use substitutions so even if the environment was producing stuff it wouldn't get pulled, so some fixes would be needed in the poll handling.

Yeah, if you could make a fresh issue, that would be great.

@cdent
Copy link
Owner

cdent commented Jun 10, 2016

This jiggery pokery in #152 to make sure that the pytest filtering doesn't interfere with non gabbi tests loaded in the same session makes an assumption that the yaml files do not contain underscores so might be wise to codify that and warn when the filenames don't match.

cdent added a commit that referenced this issue Jun 16, 2016
TestSuites are grouped by test runners by the name that is
generated for the tests within. To ease that grouping it is
useful to have filenames with a known set of characters.

Grouping is also used for py.test fixture handling.

This change raises a warning if a filename has an '_' in it,
effectively declaring that though it is possible to use '_',
it can impact grouping.

Fixes #144
cdent added a commit that referenced this issue Jun 16, 2016
TestSuites are grouped by test runners by the name that is
generated for the tests within. To ease that grouping it is
useful to have filenames with a known set of characters.

Grouping is also used for py.test fixture handling.

This change raises a warning if a filename has an '_' in it,
effectively declaring that though it is possible to use '_',
it can impact grouping.

Fixes #144
@cdent
Copy link
Owner

cdent commented Aug 16, 2016

re-opening this to make it clear that this question should be added to the faq being started in #163

@cdent cdent reopened this Aug 16, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant