Видео 124
Просмотров 4 595

OSG Training: Moving your data with Pelican (and the OSDF)

48:23

HTC23 Final Closing Remarks

12:49

HTC23 Generalizing HEPCloud Split Starter for Multiple Sites and Experiments

17:32

HTC23 E4S: Extreme-scale Scientific Software Stack

17:54

HTC23 Scientific Workflows with Pegasus

20:21

HTC23 The Condor hatches in Africa

19:35

OSG Training: Workflows with Pegasus

An introduction to the Pegasus Workflow Management System, which is a useful tool for researchers needing to execute a large number of jobs or complex workflows. Attendees will learn how to construct and manage workflows, capabilities like automatic data transfers, and higher level tooling to analyze the workflow performance.

Видео

OSG Training: Moving your data with Pelican (and the OSDF)

48:23

OSG Training: Moving your data with Pelican (and the OSDF)

Просмотров 485 месяцев назад

The OSPool runs hundreds of thousands of jobs everyday, and transferring data can sometimes require lots of power and effort. This training video covers how to compute and transfer large data using the OSDF, and how to use pelican to create data federations. Recorded June 2024.

12:49

HTC23 Final Closing Remarks

Просмотров 13Год назад

To close HTC23, Miron Livny gives some final closing remarks. Slides: agenda.hep.wisc.edu/event/2014/contributions/28451/ All talks, speakers, and materials from HTC23: agenda.hep.wisc.edu/event/2014/contributions/ All materials from past HTC Weeks and European Workshop Meetings: htcondor.org/past_condor_week...

HTC23 Generalizing HEPCloud Split Starter for Multiple Sites and Experiments

17:32

HTC23 Generalizing HEPCloud Split Starter for Multiple Sites and Experiments

Просмотров 13Год назад

Bruno Coimbra and his team discuss generalizing HEPCloud Split Starter (HPSS). They explain what HPSS is, why it's needed, and implementation. Coimbra and his team then talk about running test jobs on Summit (OLCF). Templating, batch system scripts, VO repositories, and VO account.conf are also covered. Slides: agenda.hep.wisc.edu/event/2014/contributions/28450/ All talks, speakers, and materia...

HTC23 E4S: Extreme-scale Scientific Software Stack

17:54

HTC23 E4S: Extreme-scale Scientific Software Stack

Просмотров 20Год назад

Professor Sameer Shende discusses the Extreme-scale Scientific Software Stack (E4S) and the Exascale Computing Project. They then break down how to get started with E4S on OSG-HTC and aspects E4S includes. They delve into key considerations when using E4S and support. Slides: agenda.hep.wisc.edu/event/2014/contributions/28449/ All talks, speakers, and materials from HTC23: agenda.hep.wisc.edu/e...

20:21

HTC23 Scientific Workflows with Pegasus

Просмотров 28Год назад

Karan Vahi discusses scientific workflows with Pegasus. They explain what workflows are and challenges of them across domains. Vahi also breaks down key Pegasus concepts and Pegasus's part in the ACCESS support strategy. Slides: agenda.hep.wisc.edu/event/2014/contributions/28448/ All talks, speakers, and materials from HTC23: agenda.hep.wisc.edu/event/2014/contributions/ All materials from past...

19:35

HTC23 The Condor hatches in Africa

Просмотров 14Год назад

Marco van Zwetselaar explains the arrival of HTCondor in Africa. He explains how his team was able to get two hpcs up and running in Africa, as well as the benefits that come with that. Slides: agenda.hep.wisc.edu/event/2014/contributions/28447/ All talks, speakers, and materials from HTC23: agenda.hep.wisc.edu/event/2014/contributions/ All materials from past HTC Weeks and European Workshop Me...

HTC23 Transitioning Users to SciTokens and Getting them Closer to HTCondor with Jobsub_lite

21:43

HTC23 Transitioning Users to SciTokens and Getting them Closer to HTCondor with Jobsub_lite

Просмотров 8Год назад

Shreyas Bhat provides a brief background on the jobsub project and past issues. They then dive into an overview on jobsub_lite, tokens in the software, adoption of the software, and sucesses. They end with relaying the lessons they've learned during this transition to SciTokens. Slides: agenda.hep.wisc.edu/event/2014/contributions/28446/ All talks, speakers, and materials from HTC23: agenda.hep...

HTC23 Tracing Service for Tracing-Driven Glidein Optimizations

16:31

HTC23 Tracing Service for Tracing-Driven Glidein Optimizations

Просмотров 5Год назад

Namratha Urs and her team discuss GildeinWMS and the current state of Telemetry in GWMS. They also explain motivations and possible solutions, as well as how distributed tracing works. The team then breaks down the GWMS tracing service architecture. Slides: agenda.hep.wisc.edu/event/2014/contributions/28445/ All talks, speakers, and materials from HTC23: agenda.hep.wisc.edu/event/2014/contribut...

HTC23 Refactoring GlideinWMS Credentials

14:51

HTC23 Refactoring GlideinWMS Credentials

Просмотров 9Год назад

Bruno Coimbra and his team discuss refactoring GildeinWMS credentials. They go over what GildeinWMS is as well as the related factory and frontend. Traditional X509 Authentication, token authentication, credential refactoring, and generic credentials are also covered. Slides: agenda.hep.wisc.edu/event/2014/contributions/28444/ All talks, speakers, and materials from HTC23: agenda.hep.wisc.edu/e...

14:53

HTC23 HTCondor at BNL SDCC

Просмотров 6Год назад

Kavin Casella, Tom Smith, and Christopher Hollowell discuss the Scientific Data and Computing Center (SDCC) at Brookhaven National Laboratory (BNL). They provide a brief overview of the center and relay how HTC is incorporated into the center. Slides: agenda.hep.wisc.edu/event/2014/contributions/28442/ All talks, speakers, and materials from HTC23: agenda.hep.wisc.edu/event/2014/contributions/ ...

19:55

HTC23 HTCondor at Fermilab

Просмотров 14Год назад

Farrukh Khan discusses HTCondor at Fermilab. He explains what the Fermi National Accelerator Laboratory (Fermilab) is, as well as HTCondor's function in the lab. Khan then explains the CMS and Dune global pools, as well as different facilities in Fermilab. Slides: agenda.hep.wisc.edu/event/2014/contributions/28441/ All talks, speakers, and materials from HTC23: agenda.hep.wisc.edu/event/2014/co...

HTC23 Reaching new scales in the CMS Global pool

13:53

HTC23 Reaching new scales in the CMS Global pool

Просмотров 3Год назад

Marco Mascheroni and his team discuss the CMS SI team evaluating scalability and why they do so. They also present the 2023 scalability tests they did and explain what it means. They end off with reasons as to why they must keep pushing the SI for higher scales. Slides: agenda.hep.wisc.edu/event/2014/contributions/28440/ All talks, speakers, and materials from HTC23: agenda.hep.wisc.edu/event/2...

HTC23 Intel Microarchitecture: Nomen Numen

19:13

HTC23 Intel Microarchitecture: Nomen Numen

Просмотров 7Год назад

Greg Thain discusses Intel Microachitecture and the importance of well-named terms. He delves into the powers of clarity, metaphor, and generation when naming complex concepts. Thain then applies this to the OSPool as well as how Intel and RedHat help fix this problem. Slides: agenda.hep.wisc.edu/event/2014/contributions/28439/ All talks, speakers, and materials from HTC23: agenda.hep.wisc.edu/...

HTC23 Developing Machine Learning for Metadata Standardization with HTCondor and Snakemake

17:02

HTC23 Developing Machine Learning for Metadata Standardization with HTCondor and Snakemake

Просмотров 23Год назад

Yuriy Sverchkov, Mark Craven, and Colin Dewey discuss the development of machine learning for metadata standardization using HTCondor and Snakemake. They provde some useful context on metadata and the task of standardizing it. Development workflow is also covered, as well as the strengths and weaknesses of using Snakemake and HTCondor. Slides: agenda.hep.wisc.edu/event/2014/contributions/28432/...