The Center for High Throughput Computing Fellows Program
-Applications for the Center for High Throughput Computing (CHTC) Fellows Program for Summer 2024 are now open. The final deadline for applications is March 1, 2024 (end of the day, any time zone).
+Applications for the Center for High Throughput Computing (CHTC) Fellows Program for Summer 2024 are now open. The final deadline for applications is March 8, 2024 (end of the day, any time zone).
The CHTC Fellows Program trains students in the development and use of cyberinfrastructure through a 12-week summer program where participants will work with mentors on delivering a project that will make an impact on the nation’s science.
@@ -289,7 +289,7 @@S
How it Works
- Fellows work with a mentor to develop a project relevant to one of the areas listed. Fellows will receive a monthly stipend for participating in the Program, during their fellowship. In-person participation at CHTC is preferred to maximize interaction with mentors and others in the cohort, but remote involvement is an option for some projects for those based outside of the Madison, Wisconsin area.
+ Fellows work with a mentor to develop a project relevant to one of the areas listed. Fellows will receive a monthly stipend for participating in the Program, during their fellowship. In-person participation at CHTC is preferred to maximize interaction with mentors and others in the cohort, but remote involvement is an option for some projects for those based outside of the Madison, Wisconsin area. However, you must be located in the United States during the Fellowship Program to participate.
Eligibility:
@@ -297,15 +297,16 @@ How it Works
You must be enrolled in an undergraduate or graduate program at an accredited University or College within the US.
You must have completed at least 1 academic year by the start of the Fellowship.
U.S. citizenship is not required to participate.
+ In order to ensure the safety of the workplace, proof of COVID-19 vaccination is required unless an exemption is granted for medical, disability or religious reasons.
Duration of fellowship and total stipend:
- - The Fellowship Program lasts for 12 weeks during Summer 2024.
+ - The Fellowship Program lasts for 12 weeks during Summer 2024. The Program will begin on June 3 and end on August 23, 2024.
- The Program provides $650/week stipend for large projects and $325/week stipend for medium projects. The stipends will be dispersed monthly.
- It is expected that a Fellow working on a large project is available full time during the fellowship period and will not have another significant activity (such as classes, another trainee position or a job) in the same time period.
- - Applications are now open - deadline 1 March, 2024 (any time zone).
+ - Applications are now open - deadline 8 March, 2024 (any time zone).
Developing a project
@@ -331,11 +332,11 @@ Applying
Timeline:
- - Friday 1 March, 2024 - Final deadline for applications (applications will be reviewed on a rolling basis as they arrive.)
+ - Friday 8 March, 2024 - Final deadline for applications (applications will be reviewed on a rolling basis as they arrive.)
- Friday 15 March, 2024 - End of selection period for applications. Those selected will be matched to work with mentors on developing a proposal - this may involve a short interview and other follow-up. (Interviews for selected applicants will occur on a rolling basis.)
- Monday 1 April, 2024 - Deadline for submission of proposals. (Proposals will be reviewed on a rolling basis.)
- By 15 April, 2023 - Final selection of Fellows for summer 2024
- - May-Sep - Fellows work on projects. (Exact dates may depend on specific academic year constraints.)
+ - June-August - Fellows work on projects.
Funding
diff --git a/preview-update-submodules/fellowships/list.html b/preview-update-submodules/fellowships/list.html
index 33fd60412..09eb17d14 100644
--- a/preview-update-submodules/fellowships/list.html
+++ b/preview-update-submodules/fellowships/list.html
@@ -269,7 +269,7 @@ Software Development
- User interface and Jupyter widgets
+ Synchronizing datasets via the OSDF
Large
@@ -278,40 +278,14 @@ User i
- Historically, the User Interface to HTCondor has been purely through the command-line.
-This often presents a large hurdle for onboarding new users, both in terms of even
-getting to the command-line via ssh and some terminal emulator, and then using the
-command line. Modern notebook interfaces, such as Jupyter have allowed us to minimize
-the first problem, by providing a web interface to the terminal. However, users still
-need to type in commands to access the batch system. This project extends HTCondor by
-implementing various Jupyter UI widgets to visualize users’ jobs within the system, and
-further interface with the system. After completing this project, the student will have
-Jupyter widgets to show in their portfolio, professional JavaScript experience, and
-understanding of how a distributed High Throughput System works.
-
-
-
-
-
-
-
- Integrating Pytorch and Pelican
-
-
-
-
-
-
-
-
- PyTorch is one of the most popular machine learning frameworks. An important aspect of
-using it is the data engineering: how is input data fed into the model during training?
-Going from “tutorial scale” problems to cutting-edge research requires drastically different
-techniques around data handling.
+ The pelican client tool provides a standalone mechanism to upload and download single
+files from a data federation. While the client works well, a typical use case is downloading
+an entire dataset - potentially, tens of thousands of files or hundreds of terabytes.
+When moving a dataset, features like state management and resumption become important.
-For this project, we aim to better integrate Pelican into the PyTorch community, providing both
-technical mechanisms (implementing the fsspec interface for Pelican) and documentation by providing
-tutorials and recipes for scaling PyTorch-based training using a combination of HTCondor and Pelican.
+In this project, we will explore better ways to automate dataset movement to and from the
+OSDF, exploring approaches like the HTCondor file transfer tool.
+The goal for the end of the summer would be to move a 100TB dataset using a single command.
@@ -319,23 +293,23 @@ Integrati
- ML for failure classification in the OSPool
+ Database techniques and data architecture
- Large
+ Medium
- The OSPool runs hundreds of thousands of jobs every day on dozens of
-different sites, each unique in their own way. Naturally, there are
-many hundreds of failures, most of which the system works around, but
-with added latency to workflow completion. This project would attempt
-to automatically classify failures from job logs to detect common
-patterns and highlight places for humans to look to fix common failures
-with the most payoff. Students working on this project will gain
-experience applying ML techniques to real world problems.
+ The HTCondor scheduler stores historical information about completed and
+removed jobs in a single, flat, sequential file on disk with little indexing.
+As such, querying this file for information about a single job is very slow.
+This project will add an index, perhaps with an off-the-shelf open source
+database or on-disk hashing tool to provide quick access to single entries
+in the history, and use this index in the python based cli tool. Students
+completing this project will gain real world experience in database
+techniques and Python programming.
@@ -343,7 +317,7 @@
- Distributed training of ML models
+ Integrating Pytorch and Pelican
Large
@@ -352,12 +326,14 @@ Distrib
- As machine learning models become larger and more complex, the computational needs expand beyond
-single-GPU and single-machine capabilities. This project aims to leverage distributed ML software
-(e.g. PyTorch elastic) within the HTCondor Software Suite (HTCSS) to utilize CHTC resources in
-these large computing tasks. Students will develop strategies, software components, and guides
-to enable researchers to distribute training tasks across multiple GPU nodes within the cluster.
-Students working on this project will develop valuable ML and distributed computing knowledge.
+ PyTorch is one of the most popular machine learning frameworks. An important aspect of
+using it is the data engineering: how is input data fed into the model during training?
+Going from “tutorial scale” problems to cutting-edge research requires drastically different
+techniques around data handling.
+
+For this project, we aim to better integrate Pelican into the PyTorch community, providing both
+technical mechanisms (implementing the fsspec interface for Pelican) and documentation by providing
+tutorials and recipes for scaling PyTorch-based training using a combination of HTCondor and Pelican.
@@ -368,7 +344,7 @@ Distrib
Benchmarking OSDF Performance via the OSPool
-
+ Large
@@ -390,30 +366,7 @@
- Enhancing container image build system
-
-
- Medium
-
-
-
-
-
-
Container images are a widely used technology to package and distribute
-software and services for use in systems such as Docker or Kubernetes.
-The PATh project builds hundreds of these images on a weekly basis but
-the build system needs improvement to support more images and additional
-use cases. This project will focus on taking the existing system and
-adding configurable, per-image build options. After a successful summer,
-the student fellow will gain skills in Docker containers, GitHub actions, and Bash.
-
-
-
-
-
-
-
- Grid Exerciser
+ User interface and Jupyter widgets
Large
@@ -422,14 +375,16 @@ Grid Exerciser
- The OSPool is a very large, very dynamic, heterogeneous high throughput system composed of execute
-points from dozens of campuses all over the United States. Sometimes, something will go wrong
-at one of these many sites, or one network, or one storage point, and it is difficult to determine
-where the problem is. This project proposed the design and construction of a “Grid Exerciser”,
-which consists of intentionally sending sample jobs to targeted locations on the OSPool to verify
-correct operation and sufficient performance. The project will also have a reporting and
-visualization component so that the voluminous results can be understood by a human in a
-concise manner.
+ Historically, the User Interface to HTCondor has been purely through the command-line.
+This often presents a large hurdle for onboarding new users, both in terms of even
+getting to the command-line via ssh and some terminal emulator, and then using the
+command line. Modern notebook interfaces, such as Jupyter have allowed us to minimize
+the first problem, by providing a web interface to the terminal. However, users still
+need to type in commands to access the batch system. This project extends HTCondor by
+implementing various Jupyter UI widgets to visualize users’ jobs within the system, and
+further interface with the system. After completing this project, the student will have
+Jupyter widgets to show in their portfolio, professional JavaScript experience, and
+understanding of how a distributed High Throughput System works.
@@ -437,23 +392,21 @@ Grid Exerciser
- Database techniques and data architecture
+ Distributed training of ML models
- Medium
+ Large
- The HTCondor scheduler stores historical information about completed and
-removed jobs in a single, flat, sequential file on disk with little indexing.
-As such, querying this file for information about a single job is very slow.
-This project will add an index, perhaps with an off-the-shelf open source
-database or on-disk hashing tool to provide quick access to single entries
-in the history, and use this index in the python based cli tool. Students
-completing this project will gain real world experience in database
-techniques and Python programming.
+ As machine learning models become larger and more complex, the computational needs expand beyond
+single-GPU and single-machine capabilities. This project aims to leverage distributed ML software
+(e.g. PyTorch elastic) within the HTCondor Software Suite (HTCSS) to utilize CHTC resources in
+these large computing tasks. Students will develop strategies, software components, and guides
+to enable researchers to distribute training tasks across multiple GPU nodes within the cluster.
+Students working on this project will develop valuable ML and distributed computing knowledge.
@@ -461,23 +414,23 @@
- Synchronizing datasets via the OSDF
+ Grid Exerciser
-
+ Large
-
The pelican client tool provides a standalone mechanism to upload and download single
-files from a data federation. While the client works well, a typical use case is downloading
-an entire dataset - potentially, tens of thousands of files or hundreds of terabytes.
-When moving a dataset, features like state management and resumption become important.
-
-In this project, we will explore better ways to automate dataset movement to and from the
-OSDF, exploring approaches like the HTCondor file transfer tool.
-The goal for the end of the summer would be to move a 100TB dataset using a single command.
+ The OSPool is a very large, very dynamic, heterogeneous high throughput system composed of execute
+points from dozens of campuses all over the United States. Sometimes, something will go wrong
+at one of these many sites, or one network, or one storage point, and it is difficult to determine
+where the problem is. This project proposed the design and construction of a “Grid Exerciser”,
+which consists of intentionally sending sample jobs to targeted locations on the OSPool to verify
+correct operation and sufficient performance. The project will also have a reporting and
+visualization component so that the voluminous results can be understood by a human in a
+concise manner.
@@ -514,7 +467,7 @@ Research Facilitation
- Easy Code Profiling
+ Developing software recipes
Medium
@@ -523,9 +476,9 @@ Easy Code Profiling
- Develop and publish recommendations for how researchers can better profile and understand
-their code. Run 1-2 focus groups to gather ideas and get feedback, and then organize a
-short workshop sharing recommendations and providing hands-on examples.
+ Implement a repository of apptainer container recipes for common software packages.
+Create a self-serve tutorial on building containers and show how to use the recipe
+repository and OSPool guides to build your own container.
@@ -552,7 +505,7 @@ Building trai
- Developing software recipes
+ Easy Code Profiling
Medium
@@ -561,9 +514,9 @@ Developing so
- Implement a repository of apptainer container recipes for common software packages.
-Create a self-serve tutorial on building containers and show how to use the recipe
-repository and OSPool guides to build your own container.
+ Develop and publish recommendations for how researchers can better profile and understand
+their code. Run 1-2 focus groups to gather ideas and get feedback, and then organize a
+short workshop sharing recommendations and providing hands-on examples.
@@ -579,31 +532,27 @@ Developing so
Fellowships
- - User interface and Jupyter widgets
-
- - Tracking server inventory and elevation
+ - Developing software recipes
- - Integrating Pytorch and Pelican
+ - Synchronizing datasets via the OSDF
- - ML for failure classification in the OSPool
+ - Database techniques and data architecture
- - Distributed training of ML models
+ - Integrating Pytorch and Pelican
- Benchmarking OSDF Performance via the OSPool
- - Easy Code Profiling
-
- Building training materials
- - Enhancing container image build system
+ - Tracking server inventory and elevation
- - Grid Exerciser
+ - User interface and Jupyter widgets
- - Database techniques and data architecture
+ - Distributed training of ML models
- - Synchronizing datasets via the OSDF
+ - Grid Exerciser
- - Developing software recipes
+ - Easy Code Profiling
diff --git a/preview-update-submodules/internships.html b/preview-update-submodules/internships.html
index d659fed95..19af09024 100644
--- a/preview-update-submodules/internships.html
+++ b/preview-update-submodules/internships.html
@@ -297,6 +297,78 @@ We Value Diversity
+ Student Hourly Positions (Undergrad and Grad)
+
+ We're always looking for smart motivated students to partner with
+ software developer and system administrator mentors. We expect students
+ to work between 10 and 20 hours a week, with 10 of those being during
+ business hours, with some flexibility on remaining hours. During the
+ summer and breaks, it is possible to work up to 29 hours per week.
+
+
+
+
+
+
+ Systems Administrator Intern
+
+
+ Internship - Morgridge Institute for Research
+
+
+
+
+
+ The Research Computing group at the Morgridge Institute for Research partners with a
+broad range of scientists to provide computing services for the Institute, the
+UW-Madison campus as part of the Center for High Throughput Computing, and the
+nation. We are seeking students looking to work less than 30 hours per week
+starting June 2024 to assist our systems administrators and to gain valuable
+training and education on state-of-the-art hardware, complex systems and leading
+technologies. This position offers high value learning opportunities in a collaborative,
+flexible and supportive working environment. The is an excellent career building
+position for candidates interested in learning a diverse set of technologies.
+CHTC manages over 500 Linux servers and network devices, providing services such
+as Kubernetes, distributed file systems, batch systems, and databases.
+
+
+
+
+
+
+
+
+
+
+ Student Science Writer
+
+
+ Internship - Morgridge Institute for Research
+
+
+
+
+
+ The NSF Partnership to Advance Throughput Computing (PATh) is looking for a student science writer to promote the impacts of the project on research and campuses across the country.
+
+Duties include:
+
+ - Feature Writing: Produce 1 long-form and a few short stories per month for web and newsletter distribution, highlighting the impacts and achievements of PATh and other related science projects. The writer will interview researchers and other story subjects from across the country, as well as relevant PATh staff. Duties also include drafting and proofing copy, conduct fact-checking, contributing to the editorial and social media strategy and copy-editing of other stories.
+ - Website Publishing: Edit and/or create webpages in support of the above across websites relevant to PATh and related projects. Contribute ideas for website content and strategy.
+ - Social Media: Draft social media posts to promote PATh efforts and the research it impacts. Assist with monitoring, analyzing, and reporting on audience engagement and related metrics across social media outlets such as X (Twitter), LinkedIn, etc.
+
+
+
+
+
+
+
+
+
@@ -306,6 +378,10 @@ We Value Diversity
Open Positions
diff --git a/preview-update-submodules/news/images/cole_cage.jpg b/preview-update-submodules/news/images/cole_cage.jpg
new file mode 100644
index 000000000..068b017ad
Binary files /dev/null and b/preview-update-submodules/news/images/cole_cage.jpg differ
diff --git a/preview-update-submodules/news/images/cole_math.jpg b/preview-update-submodules/news/images/cole_math.jpg
new file mode 100644
index 000000000..aa007d6e3
Binary files /dev/null and b/preview-update-submodules/news/images/cole_math.jpg differ
diff --git a/preview-update-submodules/news/images/cole_terrace.jpg b/preview-update-submodules/news/images/cole_terrace.jpg
new file mode 100644
index 000000000..e1dc6a609
Binary files /dev/null and b/preview-update-submodules/news/images/cole_terrace.jpg differ
diff --git a/preview-update-submodules/projects.html b/preview-update-submodules/projects.html
index 5931f3918..73030e3c5 100644
--- a/preview-update-submodules/projects.html
+++ b/preview-update-submodules/projects.html
@@ -300,32 +300,32 @@
- Natalia de Leon
+ xDD project and Shanan Peters
- Natalia de Leon, Professor of Agronomy, Department of Agronomy.
The goal of her research is to identify efficient mechanisms to better understand the genetic constitution of economically relevant traits and to improve plant breeding efficiency. Her research integrates genomic, phenomic, and environmental information to accelerate translational research for enhanced sustainable crop productivity.
+ Shanan Peters, project lead for xDD, Dean L. Morgridge Professor of Geology, Department of Geoscience.
Shanan’s primary research thrust involves quantifying the spatial and temporal distribution of rocks in the Earth’s crust in order to constrain the long-term evolution of life and Earth’s surface environment. Compiling data from scientific publications is a key component of this work and Peters and his collaborators are developing machine reading systems deployed over the xDD digital library ad cyberinfrastructure hosted in the CHTC for this purpose.
- 
+ 
- xDD project and Shanan Peters
+ Natalia de Leon
- Shanan Peters, project lead for xDD, Dean L. Morgridge Professor of Geology, Department of Geoscience.
Shanan’s primary research thrust involves quantifying the spatial and temporal distribution of rocks in the Earth’s crust in order to constrain the long-term evolution of life and Earth’s surface environment. Compiling data from scientific publications is a key component of this work and Peters and his collaborators are developing machine reading systems deployed over the xDD digital library ad cyberinfrastructure hosted in the CHTC for this purpose.
+ Natalia de Leon, Professor of Agronomy, Department of Agronomy.
The goal of her research is to identify efficient mechanisms to better understand the genetic constitution of economically relevant traits and to improve plant breeding efficiency. Her research integrates genomic, phenomic, and environmental information to accelerate translational research for enhanced sustainable crop productivity.
-
+
@@ -380,76 +380,76 @@
- Phil Townsend
+ Barry Van Veen
- Professor Phil Townsend of Forestry and Wildlife Ecology says Our research (NASA & USDA Forest Service funded) strives to understand the outbreak dynamic of major forest insect pests in North America through simulation modeling. As part of this effort, we map forest species and their abundance using multi-temporal Landsat satellite data. My colleagues have written an automatic variable selection routine in MATLAB to preselect the most important image variables to model and map forest species abundance. However, depending on the number of records and the initial variables, this process can take weeks to run. Hence, we seek resources to speed up this process.
+ The bio-signal processing laboratory develops statistical signal processing methods for biomedical problems. We use CHTC for casual network modeling of brain electrical activity. We develop methods for identifying network models from noninvasive measures of electric/ magnetic fields at the scalp, or invasive measures of the electric fields at or in the cortex, such as electrocorticography. Model identification involves high throughput computing applied to large datasets consisting of hundreds of spatial channels each containing thousands of time samples.
- 
+ 
- Biomagnetic Resonance Data Bank
+ Atlas Experiment
- The Biomagnetic Resonance Data Bank (BMRB) is headquarted within UW-Madison's National Magnetic Resonance Facility at Madison (NMRFAM) and uses the CHTC for research in connection with the Biological Magnetic Resonance Data Bank (BMRB).
-
+ Atlas Experiment
- 
+ 
- Barry Van Veen
+ Biomagnetic Resonance Data Bank
- The bio-signal processing laboratory develops statistical signal processing methods for biomedical problems. We use CHTC for casual network modeling of brain electrical activity. We develop methods for identifying network models from noninvasive measures of electric/ magnetic fields at the scalp, or invasive measures of the electric fields at or in the cortex, such as electrocorticography. Model identification involves high throughput computing applied to large datasets consisting of hundreds of spatial channels each containing thousands of time samples.
+ The Biomagnetic Resonance Data Bank (BMRB) is headquarted within UW-Madison's National Magnetic Resonance Facility at Madison (NMRFAM) and uses the CHTC for research in connection with the Biological Magnetic Resonance Data Bank (BMRB).
+
-
+
- Atlas Experiment
+ CMS LHC Compact Muon Solenoid
- Atlas Experiment
+ The UW team participating in the Compact Muon Solenoid (CMS) experiment analyzes petabytes of data from proton-proton collisions in the Large Hadron Collider (LHC). We use the unprecedented energies of the LHC to study Higgs Boson signatures, Electroweak Physics, and the possibility of exotic particles beyond the Standard Model of Particle Physics. Important calculations are also performed to better tune the experiment's trigger system, which is responsible for making nanosecond-scale decisions about which collisions in the LHC should be recorded for further analysis.
-
+ 
- CMS LHC Compact Muon Solenoid
+ Phil Townsend
- The UW team participating in the Compact Muon Solenoid (CMS) experiment analyzes petabytes of data from proton-proton collisions in the Large Hadron Collider (LHC). We use the unprecedented energies of the LHC to study Higgs Boson signatures, Electroweak Physics, and the possibility of exotic particles beyond the Standard Model of Particle Physics. Important calculations are also performed to better tune the experiment's trigger system, which is responsible for making nanosecond-scale decisions about which collisions in the LHC should be recorded for further analysis.
+ Professor Phil Townsend of Forestry and Wildlife Ecology says Our research (NASA & USDA Forest Service funded) strives to understand the outbreak dynamic of major forest insect pests in North America through simulation modeling. As part of this effort, we map forest species and their abundance using multi-temporal Landsat satellite data. My colleagues have written an automatic variable selection routine in MATLAB to preselect the most important image variables to model and map forest species abundance. However, depending on the number of records and the initial variables, this process can take weeks to run. Hence, we seek resources to speed up this process.
-
+
-
-
-
-
- User interface and Jupyter widgets
+Synchronizing datasets via the OSDF
Large
@@ -278,40 +278,14 @@
User i
Historically, the User Interface to HTCondor has been purely through the command-line. -This often presents a large hurdle for onboarding new users, both in terms of even -getting to the command-line via ssh and some terminal emulator, and then using the -command line. Modern notebook interfaces, such as Jupyter have allowed us to minimize -the first problem, by providing a web interface to the terminal. However, users still -need to type in commands to access the batch system. This project extends HTCondor by -implementing various Jupyter UI widgets to visualize users’ jobs within the system, and -further interface with the system. After completing this project, the student will have -Jupyter widgets to show in their portfolio, professional JavaScript experience, and -understanding of how a distributed High Throughput System works.
- - -
-
@@ -319,23 +293,23 @@
-
- Integrating Pytorch and Pelican
-
-
-
-
-
-
-
- PyTorch is one of the most popular machine learning frameworks. An important aspect of
-using it is the data engineering: how is input data fed into the model during training?
-Going from “tutorial scale” problems to cutting-edge research requires drastically different
-techniques around data handling.
The pelican client tool provides a standalone mechanism to upload and download single
+files from a data federation. While the client works well, a typical use case is downloading
+an entire dataset - potentially, tens of thousands of files or hundreds of terabytes.
+When moving a dataset, features like state management and resumption become important.
For this project, we aim to better integrate Pelican into the PyTorch community, providing both -technical mechanisms (implementing the fsspec interface for Pelican) and documentation by providing -tutorials and recipes for scaling PyTorch-based training using a combination of HTCondor and Pelican.
+In this project, we will explore better ways to automate dataset movement to and from the +OSDF, exploring approaches like the HTCondor file transfer tool. +The goal for the end of the summer would be to move a 100TB dataset using a single command.
Integrati
- ML for failure classification in the OSPool
+ Database techniques and data architecture
- Large
+ Medium
- The OSPool runs hundreds of thousands of jobs every day on dozens of
-different sites, each unique in their own way. Naturally, there are
-many hundreds of failures, most of which the system works around, but
-with added latency to workflow completion. This project would attempt
-to automatically classify failures from job logs to detect common
-patterns and highlight places for humans to look to fix common failures
-with the most payoff. Students working on this project will gain
-experience applying ML techniques to real world problems.
+ The HTCondor scheduler stores historical information about completed and
+removed jobs in a single, flat, sequential file on disk with little indexing.
+As such, querying this file for information about a single job is very slow.
+This project will add an index, perhaps with an off-the-shelf open source
+database or on-disk hashing tool to provide quick access to single entries
+in the history, and use this index in the python based cli tool. Students
+completing this project will gain real world experience in database
+techniques and Python programming.
@@ -343,7 +317,7 @@
- Distributed training of ML models
+ Integrating Pytorch and Pelican
Large
@@ -352,12 +326,14 @@ Distrib
- As machine learning models become larger and more complex, the computational needs expand beyond
-single-GPU and single-machine capabilities. This project aims to leverage distributed ML software
-(e.g. PyTorch elastic) within the HTCondor Software Suite (HTCSS) to utilize CHTC resources in
-these large computing tasks. Students will develop strategies, software components, and guides
-to enable researchers to distribute training tasks across multiple GPU nodes within the cluster.
-Students working on this project will develop valuable ML and distributed computing knowledge.
+ PyTorch is one of the most popular machine learning frameworks. An important aspect of
+using it is the data engineering: how is input data fed into the model during training?
+Going from “tutorial scale” problems to cutting-edge research requires drastically different
+techniques around data handling.
+
+For this project, we aim to better integrate Pelican into the PyTorch community, providing both
+technical mechanisms (implementing the fsspec interface for Pelican) and documentation by providing
+tutorials and recipes for scaling PyTorch-based training using a combination of HTCondor and Pelican.
@@ -368,7 +344,7 @@ Distrib
Benchmarking OSDF Performance via the OSPool
-
+ Large
@@ -390,30 +366,7 @@
- Enhancing container image build system
-
-
- Medium
-
-
-
-
-
-
-
- ML for failure classification in the OSPool
+Database techniques and data architecture
- Large
+ Medium
The OSPool runs hundreds of thousands of jobs every day on dozens of -different sites, each unique in their own way. Naturally, there are -many hundreds of failures, most of which the system works around, but -with added latency to workflow completion. This project would attempt -to automatically classify failures from job logs to detect common -patterns and highlight places for humans to look to fix common failures -with the most payoff. Students working on this project will gain -experience applying ML techniques to real world problems.
+The HTCondor scheduler stores historical information about completed and +removed jobs in a single, flat, sequential file on disk with little indexing. +As such, querying this file for information about a single job is very slow. +This project will add an index, perhaps with an off-the-shelf open source +database or on-disk hashing tool to provide quick access to single entries +in the history, and use this index in the python based cli tool. Students +completing this project will gain real world experience in database +techniques and Python programming.
-
-
@@ -368,7 +344,7 @@ Distributed training of ML models
+Integrating Pytorch and Pelican
Large
@@ -352,12 +326,14 @@
Distrib
As machine learning models become larger and more complex, the computational needs expand beyond -single-GPU and single-machine capabilities. This project aims to leverage distributed ML software -(e.g. PyTorch elastic) within the HTCondor Software Suite (HTCSS) to utilize CHTC resources in -these large computing tasks. Students will develop strategies, software components, and guides -to enable researchers to distribute training tasks across multiple GPU nodes within the cluster. -Students working on this project will develop valuable ML and distributed computing knowledge.
+PyTorch is one of the most popular machine learning frameworks. An important aspect of
+using it is the data engineering: how is input data fed into the model during training?
+Going from “tutorial scale” problems to cutting-edge research requires drastically different
+techniques around data handling.
For this project, we aim to better integrate Pelican into the PyTorch community, providing both +technical mechanisms (implementing the fsspec interface for Pelican) and documentation by providing +tutorials and recipes for scaling PyTorch-based training using a combination of HTCondor and Pelican.
Distrib
Benchmarking OSDF Performance via the OSPool
-
+ Large
@@ -390,30 +366,7 @@
- Enhancing container image build system
-
-
- Medium
-
-
-
-
-
-
-
+ Large
-
- Enhancing container image build system
-
-
-
-
- Medium
-
-
- Container images are a widely used technology to package and distribute -software and services for use in systems such as Docker or Kubernetes. -The PATh project builds hundreds of these images on a weekly basis but -the build system needs improvement to support more images and additional -use cases. This project will focus on taking the existing system and -adding configurable, per-image build options. After a successful summer, -the student fellow will gain skills in Docker containers, GitHub actions, and Bash.
- - -
-
-
@@ -437,23 +392,21 @@ Grid Exerciser
+User interface and Jupyter widgets
Large
@@ -422,14 +375,16 @@
Grid Exerciser
The OSPool is a very large, very dynamic, heterogeneous high throughput system composed of execute -points from dozens of campuses all over the United States. Sometimes, something will go wrong -at one of these many sites, or one network, or one storage point, and it is difficult to determine -where the problem is. This project proposed the design and construction of a “Grid Exerciser”, -which consists of intentionally sending sample jobs to targeted locations on the OSPool to verify -correct operation and sufficient performance. The project will also have a reporting and -visualization component so that the voluminous results can be understood by a human in a -concise manner.
+Historically, the User Interface to HTCondor has been purely through the command-line. +This often presents a large hurdle for onboarding new users, both in terms of even +getting to the command-line via ssh and some terminal emulator, and then using the +command line. Modern notebook interfaces, such as Jupyter have allowed us to minimize +the first problem, by providing a web interface to the terminal. However, users still +need to type in commands to access the batch system. This project extends HTCondor by +implementing various Jupyter UI widgets to visualize users’ jobs within the system, and +further interface with the system. After completing this project, the student will have +Jupyter widgets to show in their portfolio, professional JavaScript experience, and +understanding of how a distributed High Throughput System works.
Grid Exerciser
-
- Database techniques and data architecture
+Distributed training of ML models
- Medium
+ Large
The HTCondor scheduler stores historical information about completed and -removed jobs in a single, flat, sequential file on disk with little indexing. -As such, querying this file for information about a single job is very slow. -This project will add an index, perhaps with an off-the-shelf open source -database or on-disk hashing tool to provide quick access to single entries -in the history, and use this index in the python based cli tool. Students -completing this project will gain real world experience in database -techniques and Python programming.
+As machine learning models become larger and more complex, the computational needs expand beyond +single-GPU and single-machine capabilities. This project aims to leverage distributed ML software +(e.g. PyTorch elastic) within the HTCondor Software Suite (HTCSS) to utilize CHTC resources in +these large computing tasks. Students will develop strategies, software components, and guides +to enable researchers to distribute training tasks across multiple GPU nodes within the cluster. +Students working on this project will develop valuable ML and distributed computing knowledge.
- Synchronizing datasets via the OSDF
+ Grid Exerciser
-
+ Large
-
Synchronizing datasets via the OSDF
+Grid Exerciser
-
+ Large
The pelican client tool provides a standalone mechanism to upload and download single
-files from a data federation. While the client works well, a typical use case is downloading
-an entire dataset - potentially, tens of thousands of files or hundreds of terabytes.
-When moving a dataset, features like state management and resumption become important.
In this project, we will explore better ways to automate dataset movement to and from the -OSDF, exploring approaches like the HTCondor file transfer tool. -The goal for the end of the summer would be to move a 100TB dataset using a single command.
+The OSPool is a very large, very dynamic, heterogeneous high throughput system composed of execute +points from dozens of campuses all over the United States. Sometimes, something will go wrong +at one of these many sites, or one network, or one storage point, and it is difficult to determine +where the problem is. This project proposed the design and construction of a “Grid Exerciser”, +which consists of intentionally sending sample jobs to targeted locations on the OSPool to verify +correct operation and sufficient performance. The project will also have a reporting and +visualization component so that the voluminous results can be understood by a human in a +concise manner.
-
@@ -552,7 +505,7 @@ Building trai
Easy Code Profiling
+Developing software recipes
Medium
@@ -523,9 +476,9 @@
- Easy Code Profiling
Develop and publish recommendations for how researchers can better profile and understand -their code. Run 1-2 focus groups to gather ideas and get feedback, and then organize a -short workshop sharing recommendations and providing hands-on examples.
+Implement a repository of apptainer container recipes for common software packages. +Create a self-serve tutorial on building containers and show how to use the recipe +repository and OSPool guides to build your own container.
Building trai
- Developing software recipes
+ Easy Code Profiling
Medium
@@ -561,9 +514,9 @@ Developing so
- Implement a repository of apptainer container recipes for common software packages.
-Create a self-serve tutorial on building containers and show how to use the recipe
-repository and OSPool guides to build your own container.
+ Develop and publish recommendations for how researchers can better profile and understand
+their code. Run 1-2 focus groups to gather ideas and get feedback, and then organize a
+short workshop sharing recommendations and providing hands-on examples.
@@ -579,31 +532,27 @@ Developing so
Fellowships
- - User interface and Jupyter widgets
-
- - Tracking server inventory and elevation
+ - Developing software recipes
- - Integrating Pytorch and Pelican
+ - Synchronizing datasets via the OSDF
- - ML for failure classification in the OSPool
+ - Database techniques and data architecture
- - Distributed training of ML models
+ - Integrating Pytorch and Pelican
- Benchmarking OSDF Performance via the OSPool
- - Easy Code Profiling
-
- Building training materials
- - Enhancing container image build system
+ - Tracking server inventory and elevation
- - Grid Exerciser
+ - User interface and Jupyter widgets
- - Database techniques and data architecture
+ - Distributed training of ML models
- - Synchronizing datasets via the OSDF
+ - Grid Exerciser
- - Developing software recipes
+ - Easy Code Profiling
diff --git a/preview-update-submodules/internships.html b/preview-update-submodules/internships.html
index d659fed95..19af09024 100644
--- a/preview-update-submodules/internships.html
+++ b/preview-update-submodules/internships.html
@@ -297,6 +297,78 @@ We Value Diversity
+ Student Hourly Positions (Undergrad and Grad)
+
-
-
@@ -579,31 +532,27 @@ Developing software recipes
+Easy Code Profiling
Medium
@@ -561,9 +514,9 @@
Developing so
Implement a repository of apptainer container recipes for common software packages. -Create a self-serve tutorial on building containers and show how to use the recipe -repository and OSPool guides to build your own container.
+Develop and publish recommendations for how researchers can better profile and understand +their code. Run 1-2 focus groups to gather ideas and get feedback, and then organize a +short workshop sharing recommendations and providing hands-on examples.
Developing so
Fellowships
- - User interface and Jupyter widgets
-
- - Tracking server inventory and elevation
+ - Developing software recipes
- - Integrating Pytorch and Pelican
+ - Synchronizing datasets via the OSDF
- - ML for failure classification in the OSPool
+ - Database techniques and data architecture
- - Distributed training of ML models
+ - Integrating Pytorch and Pelican
- Benchmarking OSDF Performance via the OSPool
- - Easy Code Profiling
-
- Building training materials
- - Enhancing container image build system
+ - Tracking server inventory and elevation
- - Grid Exerciser
+ - User interface and Jupyter widgets
- - Database techniques and data architecture
+ - Distributed training of ML models
- - Synchronizing datasets via the OSDF
+ - Grid Exerciser
- - Developing software recipes
+ - Easy Code Profiling
+ We're always looking for smart motivated students to partner with + software developer and system administrator mentors. We expect students + to work between 10 and 20 hours a week, with 10 of those being during + business hours, with some flexibility on remaining hours. During the + summer and breaks, it is possible to work up to 29 hours per week. +
+ + + +
+
+
+
+
+
+
+ Systems Administrator Intern
+
+
+
+
+ Internship - Morgridge Institute for Research
+
+
+ The Research Computing group at the Morgridge Institute for Research partners with a
+broad range of scientists to provide computing services for the Institute, the
+UW-Madison campus as part of the Center for High Throughput Computing, and the
+nation. We are seeking students looking to work less than 30 hours per week
+starting June 2024 to assist our systems administrators and to gain valuable
+training and education on state-of-the-art hardware, complex systems and leading
+technologies. This position offers high value learning opportunities in a collaborative,
+flexible and supportive working environment. The is an excellent career building
+position for candidates interested in learning a diverse set of technologies.
+CHTC manages over 500 Linux servers and network devices, providing services such
+as Kubernetes, distributed file systems, batch systems, and databases.
+
+
+
+
+
+ Student Science Writer
+
+
+
+
+ Internship - Morgridge Institute for Research
+
+
+ The NSF Partnership to Advance Throughput Computing (PATh) is looking for a student science writer to promote the impacts of the project on research and campuses across the country.
+ +Duties include:
+-
+
- Feature Writing: Produce 1 long-form and a few short stories per month for web and newsletter distribution, highlighting the impacts and achievements of PATh and other related science projects. The writer will interview researchers and other story subjects from across the country, as well as relevant PATh staff. Duties also include drafting and proofing copy, conduct fact-checking, contributing to the editorial and social media strategy and copy-editing of other stories. +
- Website Publishing: Edit and/or create webpages in support of the above across websites relevant to PATh and related projects. Contribute ideas for website content and strategy. +
- Social Media: Draft social media posts to promote PATh efforts and the research it impacts. Assist with monitoring, analyzing, and reporting on audience engagement and related metrics across social media outlets such as X (Twitter), LinkedIn, etc. +
@@ -306,6 +378,10 @@
We Value Diversity
Open Positions
-
Natalia de Leon
+xDD project and Shanan Peters
- Natalia de Leon, Professor of Agronomy, Department of Agronomy.
The goal of her research is to identify efficient mechanisms to better understand the genetic constitution of economically relevant traits and to improve plant breeding efficiency. Her research integrates genomic, phenomic, and environmental information to accelerate translational research for enhanced sustainable crop productivity.
+ Shanan Peters, project lead for xDD, Dean L. Morgridge Professor of Geology, Department of Geoscience.
Shanan’s primary research thrust involves quantifying the spatial and temporal distribution of rocks in the Earth’s crust in order to constrain the long-term evolution of life and Earth’s surface environment. Compiling data from scientific publications is a key component of this work and Peters and his collaborators are developing machine reading systems deployed over the xDD digital library ad cyberinfrastructure hosted in the CHTC for this purpose.
-
+
+
-
xDD project and Shanan Peters
+Natalia de Leon
- Shanan Peters, project lead for xDD, Dean L. Morgridge Professor of Geology, Department of Geoscience.
Shanan’s primary research thrust involves quantifying the spatial and temporal distribution of rocks in the Earth’s crust in order to constrain the long-term evolution of life and Earth’s surface environment. Compiling data from scientific publications is a key component of this work and Peters and his collaborators are developing machine reading systems deployed over the xDD digital library ad cyberinfrastructure hosted in the CHTC for this purpose.
+ Natalia de Leon, Professor of Agronomy, Department of Agronomy.
The goal of her research is to identify efficient mechanisms to better understand the genetic constitution of economically relevant traits and to improve plant breeding efficiency. Her research integrates genomic, phenomic, and environmental information to accelerate translational research for enhanced sustainable crop productivity.
-
+
+
-
Phil Townsend
+Barry Van Veen
- Professor Phil Townsend of Forestry and Wildlife Ecology says Our research (NASA & USDA Forest Service funded) strives to understand the outbreak dynamic of major forest insect pests in North America through simulation modeling. As part of this effort, we map forest species and their abundance using multi-temporal Landsat satellite data. My colleagues have written an automatic variable selection routine in MATLAB to preselect the most important image variables to model and map forest species abundance. However, depending on the number of records and the initial variables, this process can take weeks to run. Hence, we seek resources to speed up this process. + The bio-signal processing laboratory develops statistical signal processing methods for biomedical problems. We use CHTC for casual network modeling of brain electrical activity. We develop methods for identifying network models from noninvasive measures of electric/ magnetic fields at the scalp, or invasive measures of the electric fields at or in the cortex, such as electrocorticography. Model identification involves high throughput computing applied to large datasets consisting of hundreds of spatial channels each containing thousands of time samples.
-
+
+
-
Biomagnetic Resonance Data Bank
+Atlas Experiment
- The Biomagnetic Resonance Data Bank (BMRB) is headquarted within UW-Madison's National Magnetic Resonance Facility at Madison (NMRFAM) and uses the CHTC for research in connection with the Biological Magnetic Resonance Data Bank (BMRB). - + Atlas Experiment
-
+
+
-
Barry Van Veen
+Biomagnetic Resonance Data Bank
- The bio-signal processing laboratory develops statistical signal processing methods for biomedical problems. We use CHTC for casual network modeling of brain electrical activity. We develop methods for identifying network models from noninvasive measures of electric/ magnetic fields at the scalp, or invasive measures of the electric fields at or in the cortex, such as electrocorticography. Model identification involves high throughput computing applied to large datasets consisting of hundreds of spatial channels each containing thousands of time samples. + The Biomagnetic Resonance Data Bank (BMRB) is headquarted within UW-Madison's National Magnetic Resonance Facility at Madison (NMRFAM) and uses the CHTC for research in connection with the Biological Magnetic Resonance Data Bank (BMRB). +
-
+
+
-
Atlas Experiment
+CMS LHC Compact Muon Solenoid
- Atlas Experiment + The UW team participating in the Compact Muon Solenoid (CMS) experiment analyzes petabytes of data from proton-proton collisions in the Large Hadron Collider (LHC). We use the unprecedented energies of the LHC to study Higgs Boson signatures, Electroweak Physics, and the possibility of exotic particles beyond the Standard Model of Particle Physics. Important calculations are also performed to better tune the experiment's trigger system, which is responsible for making nanosecond-scale decisions about which collisions in the LHC should be recorded for further analysis.
-
+
+
-
CMS LHC Compact Muon Solenoid
+Phil Townsend
- The UW team participating in the Compact Muon Solenoid (CMS) experiment analyzes petabytes of data from proton-proton collisions in the Large Hadron Collider (LHC). We use the unprecedented energies of the LHC to study Higgs Boson signatures, Electroweak Physics, and the possibility of exotic particles beyond the Standard Model of Particle Physics. Important calculations are also performed to better tune the experiment's trigger system, which is responsible for making nanosecond-scale decisions about which collisions in the LHC should be recorded for further analysis. + Professor Phil Townsend of Forestry and Wildlife Ecology says Our research (NASA & USDA Forest Service funded) strives to understand the outbreak dynamic of major forest insect pests in North America through simulation modeling. As part of this effort, we map forest species and their abundance using multi-temporal Landsat satellite data. My colleagues have written an automatic variable selection routine in MATLAB to preselect the most important image variables to model and map forest species abundance. However, depending on the number of records and the initial variables, this process can take weeks to run. Hence, we seek resources to speed up this process.
-
+
+