[Epic] Research for Skills Matrix Content Audit

[kcoronel]

Overview

This is an epic issue that will serve as a directory of all the workflows related to research to be conducted on the content found in the Skills Matrix for Design, Research, Engineering, Data, Product, and DevOps Communities of Practice.

Action Items

• [ ]

Resources/Instructions

• List of CoP Leads
• List of HfLA Projects - look only at active projects
• Skills Matrix folder
• HfLA Skills and Experience Matrix
• [Epic] Research for Skills Matrix Content Audit  #567 Issue for Research on Skills Matrix content

[xxl228]

Discussed as a team and will explore the following items while we conduct the usability test in the next a few weeks.

• Rishi will research a tool for card sort which could be useful for testing the skills matrix
• Vivian will ask Karen about comparable orgs, if there is any, for us to reference.

[kcoronel]

@kcoronel to add information from Internship project

[xxl228]

Rishi found a list of tools that would be good for our purpose, but with some pros and cons. @rishi222k will share a list here in this issue. @kcoronel we will need your help to explore the best option.

We also discussed that in the usability test, we found that some participants have different ideas about how the skills are grouped, which reassures that a card sort study would help us get a better grouping of skills in the skill matrix.

[rishi222k]

@kcoronel @xxl228 Here’s a breakdown of the card sorting tools I found, with some options we can consider:

1. UserTesting.com: Offers free and discounted plans for nonprofit organizations. Might have to contact them for plan details. Requires us to apply here. (They've worked with Code for America before).

If we're able to sign up with this, I think this might be one of the better options in terms of quality and scale for our needs.

2. Since we’re a nonprofit, I’m not sure if we have access to business accounts or allowances for paid plans. Most tools I looked at have some paid limitations; the most flexible free plan options were from KardSort and UX Tweak:

• UX Tweak: Free plan allows 1 active study at a time, 20 cards per study, 30 responses/month, and 5 recordings/month.
• KardSort: Completely free, but limited to 20 cards and 10 questions per session. No limits on the number of studies.
• Userbit: Free plan offers 2 free card sorting studies, but I couldn’t find info on usage limits. Need to test it to confirm.

Due to usage limits, one approach here could be to break the study into smaller parts (e.g. by skill category or similar) and run separate tests for each. Might be a little work to manage, but we’d be able to run the sessions async and get analytics and insights for the tests.

3. Another approach could be using open canvas tools like Miro or Figjam to create card sorting templates and run moderated tests. The downside is that they won’t provide analytics, so we’d have to manually analyze the data. These would also need to be run synchronously with a moderator, and would most likely have fewer responses.

I’ve listed all the tools I researched in this doc

[xxl228]

@kcoronel We would like your assistance to apply for the discount plan for UserTesting.com. Specifics are listed in the post by Rishi. Please see point 1. Please let us know if you have any questions. We believe this would be great for Hack for LA to have as a resource long term. Therefore, it is better that you apply for it so it can be attached to a more permeant email and contact.

[kcoronel]

@rishi222k @xxl228 let's do this:

1. Please ask the UI/UX CoP if they have any specific card sorting tools that they use
2. Please gather what UserTesting.com what documentation they would need for us to apply for a free account of their tool.

Please list the message you sent to the community and date as a comment and documentation needed as another comment. Thanks!

[kcoronel]

Also, is it okay to assign this to @rishi222k as the lead?

[xxl228]

That's fine.

[xxl228]

To do for upcoming week:

• Rishi will send a message in ux-ui channel to ask CoP about tools for card sort.
• Vivian will figure out 1Password

We will reconvene next week and determine if we want to apply for UserTesting.com on behalf of HfLA.

[rishi222k]

This message was sent on the UI/UX CoP channel - 10/07/24. Thread

Hi Community! 👋

For the CivicTechJobs project, we're planning a card sorting exercise and would love recommendations on tools you've used or come across. For our needs, it would be a plus if the tool offers:

• Generous limits on cards and categories/groupings
• Flexible participant/invite capacity
• Support for asynchronous participation

If you've conducted similar exercises or have experience with any card sorting tools, we'd greatly appreciate your insights and recommendations.

Thanks in advance!

[rishi222k]

Hi @kcoronel

I did a quick look up on how we could proceed with UserTesting.com. OneWorld program

At this stage, UserTesting.com doesnt require any documentation from us. We just need to submit a Google form to apply. We also need to meet the specified criteria before applying. They've outlined it in the same form.

[rishi222k]

Hey @kcoronel !

To extend on the previous comment, @xxl228 and I were reviewing the form during our last meeting and had a few questions on the form, which could use your input:

2. For the second question, We're unsure whether the requirements change depending on how we respond to the first question. The form doesn’t mention this, and it might not even be the case. It feels like a survey-style question, where they're just asking for our input on potential requirements.

Either way, would be helpful to get your thoughts on how to approach these questions

[kcoronel]

Hey @rishi222k @xxl228 I'm trying to weigh the consequences of completing this on our own or bringing in the rest of the organization. Would you mind posting this on Admin Slack Channel (once we have a google doc with responses to these questions). If you get it done before Friday - ping me and I can bring it up at the PM meeting.

Responses below:

1. HfLA is actually an organization that is part of a larger (umbrella) entity called Civic Tech Structure. CTS is a 501C-3.

2. I think what would be most beneficial to the project is the first option? I will defer to you both.

3. I don't think we would be willing to do any of these options, so let's leave it blank.

Thanks!

[xxl228]

We discussed about the choice of continue to peruse the UserTesting.com access for HfLA. We believe that it would be the best option for us and for the future of the team, however, it is taking some time and it may not work out at the end. But we are willing to give it a try. Vivian will post the questions in the admin channel to see if we get any response.

We also discussed the alternative to this option, which is using a free plan of another platform, where Rish has previously researched. UX Tweak comes to be a second good option, but it would require some creativity on our end to make it work. We will see how the responses are from the admin channel about getting a licenses for HfLA before we determine.

[xxl228]

Hi @kcoronel @rishi222k, I posted in the admin channel about the issue of UserTesting.com, and Ryan Chase from 311 Data Project replied saying that their team is researching on the exact same issue and they have made more progress already. Bonnie is supposed to review the potential agreement with UserTesting.com next. Once the agreement goes through, all HfLA teams are able to make use it.

[xxl228]

Met with Ryan Chase from 311 on Monday 11/18, currently working with him to gather support document for Bonnie, and a potential survey from UXRs across HfLA to advocate for getting UserTesting.com license.

[xxl228]

Received feedback from Ryan Chase and plan to proceed with a Survey with UXRs at HfLA. Reached out to CoP leads to get additional support and ideas before proceeding with the Survey. Begin to draft the survey.