Technical Excellence
|
Research Scientist I | Research Scientist II | Senior Research Scientist I | Senior Research Scientist II | Staff Research Scientist |
Leadership
|
Research Scientist I | Research Scientist II | Senior Research Scientist I | Senior Research Scientist II | Staff Research Scientist |
Autonomy
|
Research Scientist I | Research Scientist II | Senior Research Scientist I | Senior Research Scientist II | Staff Research Scientist |
Teamwork
|
Research Scientist I | Research Scientist II | Senior Research Scientist I | Senior Research Scientist II | Staff Research Scientist |
| Technical Knowledge | Has a solid foundation in ML/NLP, including a working knowledge of older, non-neural approaches. Occassionally needs to consult resources for some fundamental concepts (e.g., MLE, PyTorch documentation, loss functions). |
| Technical Problem Solving | Can diagnose and fix common issues regarding ML problems and concepts; however, needs assistance and direction when results aren't progressing for weeks. Often relies on mentors and other team members to help unblock. Needs some supervision to ensure the work is soundly executed. |
| Research Knowledge | Has a limited understanding of the research literature; is familiar with a few seminal papers in the past ~3 years and popular tasks but lacks breadth, depth, and historical awareness. |
| Research Abilities | Is generally familiar with the process of conducting novel research but needs assistance in identifying problems and approaches. Often needs guidance in balancing time spent on reading papers, brainstorming novel research ideas, writing code, and running experiments. |
| Research Writing | Has limited experience publishing papers (e.g., 1-2 first-author papers). Can write sections of a paper that concerns their work but needs assistance in crafting the narrative and phrasing the paper in the style of a strong research paper. |
| Modeling/Coding | Can comfortably write ML code in PyTorch and run sound experiments. Occassionally (once every 1-2 weeks) relies on others for help in debugging tricky code. Has limited ability in brainstorming and implementing novel architectures. |
| Intellectual Curiosity | Exercises curiosity in all things. Seeks understanding of company and engineering philosophy and principles. Seeks alternative ways to solve the same problem; proactively asks for feedback or input. |
| Innovation | Shares new ideas and perspectives to solving common problems. |
| Decision Scope | Accountable for decisions made regarding completion of own daily tasks and projects as directed by manager. |
| Operational Capacity | Contributes mainly in a tactical capacity. |
| Collaboration Sphere | Works to build stable internal relationships with peers in department and immediate supervisor. |
| Conflict Resolution | With manager support, resolves conflict. |
| Self-Development | Partners with manager to set personal goals. |
| Adaptability | Establishes a growth mindset by setting and acting upon personal skills goals; discusses goals with manager. Shifts focus and priorities as directed by manager and/or company leadership. |
| Judgement | Exercises sound judgement in all things; seeks counsel from manager, aligned Senior Leader, and/or HR as needed. Seeks to be a positive force on Kensho. |
| General Problem Solving | Learns to use professional concepts. Applies company policies and procedures to resolve routine issues. |
| Bias to Action | Self-starter, but does receive detailed direction on most work. |
| Influence on Work | Primary day-to-day body of work. Limited influence beyond immediate team. |
| Resourcefulness | Seeks existing documentation. Takes full advantage of all training opportunities. Attempts to unblock self; consults teammates when stuck. |
| Delegation | Manager drives most day-to-day work. Proactively seeks to add to current body of work. |
| Prioritization | Priorities are usually determined by manager. |
| Feedback | Open to giving and receiving feedback on code review, design review, and during quarterly performance evaluations. Regularly seeks feedback from peers and managers. With manager support, takes action on areas of development. Provides constructive and actionable feedback to peers and managers. |
| Communication | Practices open and transparent communication with all levels of the organization. Clearly and concisely communicates status updates (state of work, timelines, estimates, blockers, progress, knowledge gaps, PTO/sick time) with stakeholders. Updates documentation when they identify deficiencies or gaps. Shares ideas with teammates, manager, and company leadership. |
| Participation | Attends and actively participates in all team meetings; pays close attention to details. |
| Go Team | Understands their work contributes to Kensho's overall success and does whatever necessary to see it through, without doing harm. |
| Technical Knowledge | Has a strong foundation in ML/NLP, spanning traditional and modern approaches. Generally doesn't need to rely on external resources for common ML knowledge and can meaningfully discuss important ML/NLP concepts. Stays aware of some of the latest developments (e.g., PyTorch, libraries, new models or platforms). |
| Technical Problem Solving | Can diagnose and fix most issues regarding ML problems, while occasionally relying on others to help diagnose issues and provide feedback on how to unblock. Can be generally trusted that the work was soundly executed without much help or guidance. For thoroughness, we will still double-check one's work and ask them for intricate details. |
| Research Knowledge | Has a solid understanding of modern literature, including a working knowledge of some key works from 5-10 years ago. That is, has moderate amount research knowledge in terms of breadth and depth. |
| Research Abilities | Is familiar and comfortable with the process of conducting novel research and needs little assistance in identifying problems and potential approaches. Needs some assistance and feedback in balancing time spent on reading papers, brainstorming novel research ideas, writing code, and running experiments |
| Research Writing | Has solid experience publishing papers (e.g., 3-6 first author papers in top-tier conferences). Can write the majority of a paper with a moderately clear narrative; however, greatly benefits from having feedback and edits. |
| Modeling/Coding | Can write effective, efficient ML code in PyTorch. Can develop and implement novel architectures without much difficulty. Runs sound, useful experiments and can make adjustments while minimally relying on others for suggestions. Asks others for help and feedback but usually not out of reliance; that is, can generally unblock oneself. |
| Intellectual Curiosity | Exercises curiosity in all things; encourages peers to seek answers. Demonstrates strong understanding of company and engineering philosophy and principles. Provides evidence and counter proposals to support ideas. |
| Innovation | Takes appropriate amount of risk and tries new ways of doing old things. Is learning to be comfortable with failure and seeks to learn from it. Shares lessons with peers and teammates. |
| Decision Scope | Independently determines completion path of own daily tasks and projects. |
| Operational Capacity | Supports the tactical initiatives for the relative area; seeks opportunities to lead certain initiatives. |
| Collaboration Sphere | Builds productive internal/external working relationships with department and functional leadership. |
| Conflict Resolution | Resolves most conflict independently, immediately makes manager aware of issue and proposed solution. |
| Self-Development | Independently sets and achieves personal goals; assesses own progress regularly. |
| Adaptability | Regularly asks for and acts upon feedback to continually improve and adjust personal skills goals. Assesses and shifts priorities with minimal disruption. |
| Judgement | Exercises sound judgement in all things; seeks counsel from manager, aligned Senior Leader, and/or HR as needed. Behaviors and actions demonstrate good intent and positive impact on Kensho. |
| General Problem Solving | Develops professional expertise, applies company policies and procedures to resolve a variety of issues. |
| Bias to Action | Mostly independent. Normally receives little instruction on day-to-day work, general instructions on new assignments. Seeks to independently achieve results as often as possible. May act as more experienced mentor to junior teammates. |
| Influence on Work | Primary day-to-day body of work; regularly assesses own strengths, scope of work, and level of effectiveness. May influence team based projects and tasks. |
| Resourcefulness | Seeks out and maintains existing documentation. Regularly attends all training opportunities offered by Kensho. Actively seeks additional training/learning opportunities outside of Kensho. Usually unblocks self; foresees when assistance is required. |
| Delegation | Seeks to take level-appropriate tasks from manager; offers opportunities to teammates. |
| Prioritization | Usually determines own priorities; able to manage multiple priorities. Manager may help sort out priorities if conflicts arise. |
| Feedback | Proactively provides clear, actionable feedback to peers and managers in code review, design review and during quarterly performance evaluations. Regularly seeks feedback from peers and managers. Independently takes action on areas of development. |
| Communication | Proactively shares knowledge with teammates. Proactively communicates current state of the work (and all other related matters) to stakeholders. Proactively communicates concerns and blockers to peers and/or managers as appropriate. Proactively updates documentation when updating code. |
| Participation | Actively participates in retrospectives, provide detailed estimates, detailed explanations of technical decisions (if necessary.) Observes gaps in current processes and offers suggested solutions. Offers new ideas to enhance experience of team members. |
| Go Team | Acts as an internal cheerleader to peers and teammates. Seeks out and actively engages in opportunities to further Kensho's mission (e.g. career fair ambassador, lightning talks, interview panelist, etc.) |
| Technical Knowledge | Has a strong foundation in ML/NLP that spans decades of work. Seldomly needs to consult external resources for common ML knowledge. Stays abreast of many cutting-edge technologies (e.g., within PyTorch, libraries, new models or platforms). |
| Technical Problem Solving | Comfortably approaches problems and can be trusted to make steady progress towards a solution without much assistance. Doesn't shy aware from difficult problems and demonstrates strong intuition with reasonable ideas -- while leveraging a wide breadth of first-hand knowledge and relevant work from others. |
| Research Knowledge | Has a strong working knowledge of cutting-edge research. This includes understanding key details of specific, compelling papers within a few weeks of their gaining attention within the community. Additionally, is aware of a vast amount of past works and is able to make connections across papers from different areas and time periods. |
| Research Abilities | Is very comfortable with the process of conducting novel research. Doesn't need help in identifying problems, and can do so within the constraints of an existing research agenda. Can identify future problems and a trajectory of research to explore. Can be trusted to lead the way in conducting many strong, initial approaches to solve each problem. Generally self-regulates their time well between the various, difficult stages of research. |
| Research Writing | Has strong experience publishing papers (e.g., 7+ papers in top-tier conferences). Can comfortably write an entire paper with a compelling narrative. Benefits from feedback but the paper can usually be submitted without any additional assistance. Can provide useful feedback on papers written by other Kenshins. |
| Modeling/Coding | Can quickly write effective, efficient ML code in PyTorch. Develops novel, sound architectures with ease. Runs a host of pertinent experiments and make appropriate adjustments without assistance. Asks teammates for feedback but it's generally for collaboration purposes and to strengthen everyone's work, as opposed to not being able to unblock oneself. |
| Intellectual Curiosity | Exercises curiosity in all things. Builds good relationships with teammates and stakeholders. Reinforces and shares company and engineering philosophy and principles. |
| Innovation | Assesses associated risk of failure and learning trade-offs, and seeks to fail and learn frequently and rapidly to ultimately achieve desired results. Often partners with junior teammates to increase failure/learning opportunities. |
| Decision Scope | Determines completion path of own daily tasks and projects; may influence how peers do their work. Considers ripple effects and impact (present and future) of decisions. May create alternative scenarios to help undo a bad decision. |
| Operational Capacity | Leads tactical initiatives related to an important business area or process. |
| Collaboration Sphere | Networks with key contacts outside own area of expertise. Seeks out regular opportunities to collaborate strategically with functional senior staff and leadership. |
| Conflict Resolution | Resolves most conflicts independently. May identify conflicts with which they are not directly involved within the team; understands when to seek escalation paths to management and/or HR. |
| Self-Development | Seeks to work beyond own level of comfort. Demonstrates active interest in learning more about other areas outside immediate focus. |
| Adaptability | Focuses primarily on what is within one's own control to change or influence. Seeks to understand why company or org decisions are made in order to better adapt team decisions. Follows up on team decisions by focusing on the facts and adjusts accordingly & empathetically. |
| Judgement | Exercises sound judgement in all things. Behaviors and actions set positive examples internally and proudly represent Kensho externally. |
| General Problem Solving | Having wide-ranging experience using professional concepts and company objectives to resolve complex issues in creative and effective ways. |
| Bias to Action | Mostly independent. Often acts as experienced mentor. Often consults with peers to make suggestions on critical next-step decisions. |
| Influence on Work | Determines methods and procedures on new assignments. May coordinate activities of other individual team members. |
| Resourcefulness | Prunes and maintains documentation for a project even when not actively making changes. Regularly attends internal and external training/learning opportunities; may coordinate team-wide opportunities. |
| Delegation | Assigns or seeks opportunities for teammates. Managers are consulted, but remain mostly hands off based on what make sense. |
| Prioritization | Remains focused in the face of interruptions and distractions. Proactively communicates to team/manager if timelines will slip. Runs efficient, effective, and succinct meetings. Able to accurately assess self and teammates' progress and difficulties; may assist more junior teammates in prioritizing their work. |
| Feedback | Regularly provides clear, actionable, direct and compassionate feedback to peers and managers to help them thrive. Is often sought out to provide feedback to peers; is becoming comfortable delivering constructive feedback. Constantly seeking and acting upon feedback from peers and managers. |
| Communication | Practices open and transparent communication with all levels of the organization. Regular knowledge and idea sharing with team, collaborating and discussing technical topics cross-team, frankly communicate with all project stakeholders, firm grasp of presentation skills, etiquette, and slide creation (in-person and remote). |
| Participation | Consistently involved in all team activities and conversations, often leading them. Suggests ideas to improve the team and company culture and enhance existing processes. Hosts, coordinates, and attends various team events (social or otherwise). |
| Go Team | Invests ample time to better Kensho culture and progress Kensho's vision by acting both internally and externally, and is willing to make sacrifices in order to do so. |
| Technical Knowledge | Has a strong foundation in ML/NLP that spans decades of work. Rarely needs to consult external resources for common ML knowledge. Stays abreast of many cutting-edge technologies (e.g., within PyTorch, libraries, new models or platforms) and can be relied upon to bring to our attention important, new works without being solicited. |
| Technical Problem Solving | Excels at approaching problems and can be trusted to make steady progress towards a solution without assistance. Doesn't shy aware from difficult problems and demonstrates strong intuition with valid ideas -- while leveraging a wide breadth of first-hand knowledge and relevant work from others. |
| Research Knowledge | Has a strong working knowledge of cutting-edge research. This includes understanding key details of specific, compelling papers within a few weeks of their gaining attention within the community. Additionally, is aware of a vast amount of past works and is able to make connections across papers from different areas and time periods. |
| Research Abilities | Is very comfortable with the process of conducting novel research. Doesn't need help in identifying problems, and can do so not only within the constraints of an existing research agenda but can contribute towards crafting our entire research agenda. Can identify future problems and a trajectory of research to explore. Can be trusted to lead the way in conducting many strong, initial approaches to solve each problem. Self-regulates their time well between the various, difficult stages of research. |
| Research Writing | Has strong experience publishing papers (e.g., 8+ papers in top-tier conferences). Can comfortably write an entire paper with a compelling narrative. Benefits from feedback but the paper can usually be submitted without any additional assistance. Can be relied on to provide meaningful feedback on papers written by other Kenshins. |
| Modeling/Coding | Can quickly write effective, efficient ML code in PyTorch. Develops novel, sound architectures with ease. Runs a host of pertinent experiments and make appropriate adjustments without assistance. Asks teammates for feedback but it's generally for collaboration purposes and to strengthen everyone's work, as opposed to not being able to unblock oneself. |
| Innovation | Assesses associated risk of failure and learning trade-offs, and seeks to fail and learn frequently and rapidly to ultimately achieve desired results. Frequently partners with junior teammates to increase failure/learning opportunities. |
| Collaboration Sphere | Proactively builds strong relationships with key contacts outside own area of expertise, Creates regular opportunities to collaborate strategically with functional senior staff and leadership. |
| Conflict Resolution | Is sought as a mediator to resolve technical conflicts on the team. |
| Bias to Action | Mostly independent. Often acts as experienced mentor. Often consulted by peers for suggestions on critical next-step decisions. |
| Influence on Work | Sets expectations for new projects. Coordinates activities of other individual team members. |
| Delegation | Understands and takes advantage of teammates strengths. |
| Feedback | Is often sought out by managers to provide upward feedback. |
| Technical Knowledge | Has a mastery level of ML/NLP that spans decades. Can be relied on as a subject expert for a few sub-fields/problems. Stays up-to-date and is generally aware of most of the commonly adopted new technologies within a few months of their adoption. |
| Technical Problem Solving | Demonstrates mastery in solving a wide array of problems, including complex, novel problems without known solutions. Can be relied on to leverage their wealth of deep knowledge that encompasses a wide landscape of problems. Stays up-to-date on modern technologies, tools, and works. |
| Research Knowledge | Has a mastery-level knowledge of cutting-edge research, including understanding fine-level details of the most significant papers within a few weeks/months of them gaining attention within the community. Is aware of a vast amount of past works and is able to make unique, compelling connections across papers in ways that aren't obvious to the community at large. |
| Research Abilities | Demonstrates expert-level ability in identifying important, worthwhile problems to work on, and can do so not only within the context of an existing research agenda but can help create new research agendas (i.e., a multi-year collection of projects to pursue). Needs no assistance in regulating their time. Can be trusted to not have any significant oversight within research at large (e.g., knowledge of existing works, paradigm shifts, etc). |
| Research Writing | Has extensive experience publishing papers (10+ papers in top-tier conferences, including several papers as last-author). Can comfortably write an entire paper with a compelling narrative but often leads others in doing so instead. Can provide meaningful feedback to others' papers at an early stage and throughout. |
| Modeling/Coding | Demonstrates mastery-level ability to quickly write sound, effective ML code, including novel architectures. Runs a host of useful experiments and can make correct adjustments without help. Can lead others in general best-practices. Provides rich feedback to other team members and help inform others of latest breakthroughs (e.g., models, NN frameworks, etc). |
| Intellectual Curiosity | Exercises curiosity in all things. Builds good relationships with managers and those leading products and initiatives. Understands when to be pragmatic and helps managers and those leading products and initiatives to mitigate situations where conditions are not ideal. Informs engineering philosophy and principles. |
| Innovation | Accustomed to the risks and benefits of failing and learning fast and does so frequently. Supports junior teammates through the learning process to achieve desired results. |
| Decision Scope | Makes decisions that influence and drive the work of their peers and teammates. Plans always include multiple, well-thought out contingencies. |
| Operational Capacity | Works on significant and unique issues where analysis of situations or data requires an evaluation of intangibles that is critical to the strategic planning and execution of important business processes. |
| Collaboration Sphere | Creates formal networks involving coordination among groups and builds mostly strategic relationships with peers and company leadership. |
| Conflict Resolution | Addresses and resolves/diffuses conflict proactively, before escalation is required. Attuned at identifying conflict within own team; aids in resolving/diffusing promptly, professionally. |
| Self-Development | Demonstrates ability to take on responsibilities outside their immediate area of focus. Consistently creates opportunities to challenge themselves in new ways. |
| Adaptability | Consistently reframes obstacles as opportunities. Solicits input from teams on ideas for mitigating obstacles and change. Seeks to understand historical decisions to avoid repeating mistakes and/or reassess decisions. |
| Judgement | Exercises sound, independent judgement in all things, especially with methods, techniques and evaluation criteria for obtaining results. Work/business-related decisions often reflect the notion of putting company before self. |
| General Problem Solving | Having broad expertise or unique knowledge, uses skills to contribute to development of company objectives and principles and to achieve goals in creative and effective ways. |
| Bias to Action | Acts independently to determine methods and procedures on new or special assignments. May supervise the activities of others in some capacity. Makes swift decisions based on knowns, with consideration given to the unknowns, and accepts consequences of those decisions as long as desired outcomes are achieved. |
| Influence on Work | Leverages own experience to successfully manage and direct cross team commitments and timelines. Influences work and team in a broad way, often at the departmental level. |
| Delegation | Consistently provides broader context. Increases own technical leverage by delegating responsibility over outcomes, not tasks. |
| Prioritization | Often determines the priorities and timelines of own and/or other teams. Understands and implements time effectiveness principles to keep self and others on track. |
| Feedback | Is a primary source of feedback to peers, teammates, and managers. Versed and comfortable delivering constructive feedback. Understands which elements of feedback are critical to their own success; implements changes as needed. |
| Communication | Practices open and transparent communication with all levels of the organization. Communicates deeply technical topics appropriately to a variety of audiences. Skilled in persuasive presentation (i.e. pitching the Eng function on a new technology). Demonstrable ability to utilize rhetoric and prose to inspire, build engagement, and foster high-trust teams. Competent presenting to enterprise and Kensho executives; also capable of external presentations with support. |
| Participation | Models the behavior of involvement and interaction expected within the team. Proactively shares ideas and opportunities where the team members can participate and develop their skills. Consolidates ideas from team members and brainstorms on ways to implement it at a larger scale across the organization. Supportive of and encourages participation outside of the day to day work wherever it makes sense. |
| Go Team | Embodies the team player mindset, infusing it into all actions and behaviors. Seeks to constantly better Kensho's culture and brand. |
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
