justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 232543.1@dunegpschedd02.fnal.gov

Jobsub ID232543.1@dunegpschedd02.fnal.gov
Workflow ID9307
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-24 07:36:44
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce16-multicore
Last heartbeat2025-10-24 13:44:48
From worker nodeHostnamehnode75.pic.es
cpuinfoAMD EPYC 7443P 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-24 07:37:58
Input filesmonte-carlo-009307-000001
JobscriptExit code0
Real time5h (21555s)
CPU time3h (12768s = 59%)
Max RSS bytes4404490240 (4200 MiB)
Outputting started2025-10-24 13:37:15
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09307/1/001/reco_stage1.root
Finished2025-10-24 13:44:48
Saved logsjustin-logs:232543.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

vice:prepare:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 99600 traces, 0 summary
Retagger: tagging trace set: gauss with 82406 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 82406 traces tagged "gauss"
FrameSaver: q=2.9553e+06 n=1526538 tag=gauss
wclsFrameSaver: saving 99600 traces tagged "wiener"
FrameSaver: q=3.0932e+06 n=1394393 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 23079 XUs and 21754 XVs -> 262 XUVs
C:0 T:1 35559 XUs and 52492 XVs -> 5468 XUVs
C:0 T:2 24620 XUs and 38326 XVs -> 901 XUVs
C:0 T:3 16862 XUs and 20974 XVs -> 63 XUVs
C:0 T:4 12202 XUs and 13872 XVs -> 121 XUVs
C:0 T:5 42060 XUs and 36252 XVs -> 2716 XUVs
C:0 T:6 111680 XUs and 119901 XVs -> 4601 XUVs
C:0 T:7 47350 XUs and 48824 XVs -> 574 XUVs
C:0 T:8 8720 XUs and 8376 XVs -> 93 XUVs
C:0 T:9 46188 XUs and 62838 XVs -> 3355 XUVs
C:0 T:10 20080 XUs and 13736 XVs -> 1806 XUVs
C:0 T:11 111 XUs and 77 XVs -> 11 XUVs
19971 XUVs total
7490 collection wire objects
18065 potential space points
Neighbour search...
1174909 tests to find 551282 neighbours
Iterating with no regularization...
Begin: 6.97043e+07
0 6.0128e+07
1 5.94371e+07
2 5.9328e+07
3 5.93075e+07
Now with regularization...
Begin: 5.02964e+07
0 5.02806e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410082094
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410082094
fLifetime: 30000 [us]
24-Oct-2025 15:37:13 CEST  Closed output file "reco_stage1.root"
24-Oct-2025 15:37:13 CEST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          362.209       662.145       1154.07       657.605       205.858        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00101345    0.00326484    0.00528273    0.00353997    0.00140714       10     
reco:ophit:OpHitFinder                             0.249969      0.315783      0.411418      0.294588      0.0581909       10     
reco:opflash:OpFlashFinder                          0.17114      0.212481      0.265624      0.204027      0.0300667       10     
reco:opslicer:OpSlicer                             0.555665      0.829966       1.05756      0.875983      0.158075        10     
reco:caldata:DataPrepModule                         4.47335       5.76166       6.53451       6.11671      0.729211        10     
reco:wclsdatasp:WireCellToolkit                     29.331        58.4989       281.385       35.5062       74.3586        10     
reco:gaushit:GausHitFinder                         0.959607       1.44545       1.89989       1.45219      0.314244        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000415818   0.000590671   0.000844176   0.000513764   0.000154764      10     
reco:reco3d:SpacePointSolver                        9.63398       18.3336       28.1952       18.0433       5.95095        10     
reco:hitpdune:DisambigFromSpacePoints               1.53591       2.28431       3.40775       2.30633      0.601543        10     
reco:pandora:StandardPandora                        23.2581       149.776       437.485       64.7092       136.199        10     
reco:pandoraWriter:StandardPandora                 0.339428      0.500233      0.699462      0.510747      0.110263        10     
reco:pandoraTrack:LArPandoraTrackCreation           3.08027       4.8608        7.19193       5.16576       1.45809        10     
reco:pandoraShower:LArPandoraShowerCreation         3.33948       5.7327        9.27562       5.56546       1.89578        10     
reco:pandoracalo:Calorimetry                        1.66556       2.84304       4.3954        2.73791      0.900314        10     
reco:pandoracalonosce:Calorimetry                   1.42801       2.45252       3.84775       2.42058      0.766733        10     
reco:pandorapid:Chi2ParticleID                    0.00364271    0.00660015    0.00913128    0.00736385    0.00190111       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0294377     0.0599681     0.090211      0.0637817     0.0202495       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00331155    0.00623415    0.00902401    0.00695851    0.00197331       10     
reco:pandoraShowercalo:ShowerCalorimetry            3.0013        5.55202       9.59177       5.2254        2.01356        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.62863       17.9946       132.189       4.97595       38.1384        10     
reco:emtrkmichelid:EmTrackMichelId                  172.512       298.478       517.384       271.85         114.4         10     
reco:crttag:SingleCRTMatchingProducer             0.00752959     0.0158993     0.0255337     0.015177     0.00514248       10     
reco:crtreco:TwoCRTMatchingProducer               0.00604716    0.00883505     0.0114008    0.00919672    0.00182094       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.740766       1.02287       1.45578       1.03458      0.213226        10     
reco:pandora2Track:LArPandoraTrackCreation          9.36493       13.4324       23.9343       11.3076       4.32469        10     
reco:pandora2calo:Calorimetry                       4.90451       19.3834       129.74        6.74383       36.8294        10     
reco:pandora2calonosce:Calorimetry                  4.18488       7.03654       11.8744       6.90905       2.09074        10     
reco:pandora2pid:Chi2ParticleID                    0.0109714     0.0161038     0.0245621     0.015563     0.00434399       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0491082     0.0784128     0.118966      0.0733826     0.0228041       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0115647     0.0161746     0.0238385     0.0156459    0.00401636       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.48682       4.88525        6.317        4.8372        1.07789        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.04287       31.2899       130.056       7.13913       49.208         10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.30167       5.8874        8.38211       6.51667       1.7037         10     
[art]:TriggerResults:TriggerResultInserter         2.242e-05    3.71707e-05   0.000105422   3.03405e-05   2.32646e-05      10     
end_path:out1:RootOutput                           3.86e-06     5.5722e-06     9.56e-06      5.425e-06    1.49927e-06      10     
end_path:out1:RootOutput(write)                     1.78315       3.01101       4.06328       3.14415      0.651639        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5213.94 MB
  Peak resident set size usage (VmHWM): 4404.49 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.26 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.70 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11254888
-rw-r--r-- 1 dune001 dune 2601681441 Oct 24 13:42 detsim_stage2.root
-rw-r--r-- 1 dune001 dune 2466707570 Oct 24 13:38 detsim_stage1.root
-rw-r--r-- 1 dune001 dune 2322825061 Oct 24 15:37 reco_stage1.root
-rw-r--r-- 1 dune001 dune 2063483151 Oct 24 10:17 g4_stage2.root
-rw-r--r-- 1 dune001 dune 1986225356 Oct 24 09:50 g4_stage1.root
-rw-r--r-- 1 dune001 dune   83450310 Oct 24 15:31 Pandora_Events.pndr
-rw-r--r-- 1 dune001 dune     243304 Oct 24 15:37 jobscript.log
-rw-r--r-- 1 dune001 dune     217088 Oct 24 15:37 mem.db
-rw-r--r-- 1 dune001 dune      62376 Oct 24 15:37 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 dune      36864 Oct 24 15:37 time.db
-rw-r--r-- 1 dune001 dune        519 Oct 24 13:42 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Oct 24 10:17 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Oct 24 15:37 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 dune         52 Oct 24 09:38 all-input-dids.txt
-rw-r--r-- 1 dune001 dune          7 Oct 24 15:37 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune          0 Oct 24 13:51 ProtoDUNE.log
-rw-r--r-- 1 dune001 dune          0 Oct 24 13:42 debugprod.log
justIN time: 2025-11-03 18:30:38 UTC       justIN version: 01.05.01