justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 233777.1@dunegpschedd02.fnal.gov

Jobsub ID233777.1@dunegpschedd02.fnal.gov
Workflow ID9417
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-28 15:34:20
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc03
Last heartbeat2025-10-28 18:51:48
From worker nodeHostnamewn-la-16.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-28 15:35:57
Input filesmonte-carlo-009417-000001
JobscriptExit code0
Real time3h (11663s)
CPU time3h (11500s = 98%)
Max RSS bytes4225798144 (4030 MiB)
Outputting started2025-10-28 18:50:21
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09417/1/001/reco_stage1.root
Finished2025-10-28 18:51:48
Saved logsjustin-logs:233777.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 107982 traces, 0 summary
Retagger: tagging trace set: gauss with 80777 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 80777 traces tagged "gauss"
FrameSaver: q=2.84615e+06 n=1417808 tag=gauss
wclsFrameSaver: saving 107982 traces tagged "wiener"
FrameSaver: q=3.0344e+06 n=1263193 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 19652 XUs and 24985 XVs -> 598 XUVs
C:0 T:1 59461 XUs and 92717 XVs -> 10847 XUVs
C:0 T:2 50680 XUs and 42607 XVs -> 2542 XUVs
C:0 T:3 29108 XUs and 23172 XVs -> 384 XUVs
C:0 T:4 17915 XUs and 16908 XVs -> 263 XUVs
C:0 T:5 69280 XUs and 62045 XVs -> 6395 XUVs
C:0 T:6 124006 XUs and 110949 XVs -> 15158 XUVs
C:0 T:7 46704 XUs and 41595 XVs -> 898 XUVs
C:0 T:8 4670 XUs and 5151 XVs -> 58 XUVs
C:0 T:9 25931 XUs and 38012 XVs -> 2972 XUVs
C:0 T:10 32704 XUs and 39003 XVs -> 9426 XUVs
49541 XUVs total
8596 collection wire objects
42163 potential space points
Neighbour search...
6285091 tests to find 3026262 neighbours
Iterating with no regularization...
Begin: 5.29558e+07
0 4.52093e+07
1 4.45304e+07
2 4.43979e+07
3 4.43505e+07
4 4.4326e+07
Now with regularization...
Begin: 3.68172e+07
0 3.67659e+07
1 3.67235e+07
2 3.66906e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410169613
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410169613
fLifetime: 30000 [us]
28-Oct-2025 19:50:19 CET  Closed output file "reco_stage1.root"
28-Oct-2025 19:50:19 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          263.16        328.499       474.742       297.024       69.5619        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00125351    0.00493724    0.00813848    0.00515615    0.00200251       10     
reco:ophit:OpHitFinder                             0.253134       0.29677      0.408968       0.27926      0.043159        10     
reco:opflash:OpFlashFinder                         0.148643      0.197999      0.293194      0.178226      0.0455996       10     
reco:opslicer:OpSlicer                             0.397059      0.566921      0.766405      0.553531      0.0886576       10     
reco:caldata:DataPrepModule                         4.72338       4.87396       5.42002       4.8124       0.192306        10     
reco:wclsdatasp:WireCellToolkit                     34.5584       42.8566       49.0072       42.8937       4.30898        10     
reco:gaushit:GausHitFinder                         0.841321       1.01955       1.33879      0.970365      0.152479        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000344968   0.000614903   0.00149707    0.000516929   0.000327824      10     
reco:reco3d:SpacePointSolver                        10.399        15.8539       25.5145       14.5021       5.33492        10     
reco:hitpdune:DisambigFromSpacePoints              0.821582        1.238        2.00264       1.11412      0.355583        10     
reco:pandora:StandardPandora                        17.6006       28.1194       46.8905        26.34        8.81711        10     
reco:pandoraWriter:StandardPandora                 0.322966      0.404041      0.545602      0.386761      0.0725058       10     
reco:pandoraTrack:LArPandoraTrackCreation           3.34783       5.13458       10.2232       4.69037       1.95493        10     
reco:pandoraShower:LArPandoraShowerCreation         3.23811       6.13624       14.3473       5.04489       3.18872        10     
reco:pandoracalo:Calorimetry                        1.57988       2.66411       5.66987       2.26548       1.18078        10     
reco:pandoracalonosce:Calorimetry                   1.36527       2.5352        5.07704       2.26411       1.07005        10     
reco:pandorapid:Chi2ParticleID                    0.00389054    0.00474769    0.00680284     0.0046767    0.000763443      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0331774     0.0417872     0.0564246     0.0402585    0.00683669       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00395583    0.00457164    0.00575002    0.00448758    0.000449081      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.62946       6.08581       15.778        4.94963       3.71674        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.54663       5.77269       14.5619       4.84762       3.42157        10     
reco:emtrkmichelid:EmTrackMichelId                  125.917       151.414       195.814       145.037       22.2318        10     
reco:crttag:SingleCRTMatchingProducer             0.00927486     0.016203      0.0376184     0.0142857    0.00798566       10     
reco:crtreco:TwoCRTMatchingProducer               0.00768066     0.011685      0.0267781    0.00917131    0.00562372       10     
reco:anodepiercerst0:T0RecoAnodePiercers            0.60653      0.789137       1.00155      0.748446      0.143474        10     
reco:pandora2Track:LArPandoraTrackCreation          9.67196       16.7771       35.8242       13.3384       8.71726        10     
reco:pandora2calo:Calorimetry                       3.74928       7.63199       15.8827       6.03811       4.08258        10     
reco:pandora2calonosce:Calorimetry                  3.5291         7.351        15.9486       5.90161       4.13895        10     
reco:pandora2pid:Chi2ParticleID                   0.00869223     0.0103957     0.0134818    0.00967825    0.00159719       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0424697     0.0519881     0.0685415     0.0495573    0.00908304       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.008709      0.0104679     0.0132794     0.0100212    0.00154518       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.30642       5.75109       10.526        4.66858       2.61053        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.1771        6.51489       13.8847       5.1119        3.61397        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.54291       5.61487       12.3927       4.7654        3.03866        10     
[art]:TriggerResults:TriggerResultInserter        3.4291e-05    5.14691e-05   0.000110847   4.25535e-05   2.15238e-05      10     
end_path:out1:RootOutput                           6.52e-06     7.5534e-06    1.1737e-05     7.171e-06    1.46356e-06      10     
end_path:out1:RootOutput(write)                     2.04072       2.5432        3.37771       2.38334      0.433031        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5032.29 MB
  Peak resident set size usage (VmHWM): 4225.8 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.00 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9792196
-rw-r--r--. 1 dune009 dune 2321557307 Oct 28 18:53 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2204673478 Oct 28 18:52 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 1998663485 Oct 28 19:50 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1752192581 Oct 28 16:52 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1677112131 Oct 28 16:41 g4_stage1.root
-rw-r--r--. 1 dune009 dune   72394100 Oct 28 19:47 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     242741 Oct 28 19:50 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Oct 28 19:50 mem.db
-rw-r--r--. 1 dune009 dune      62376 Oct 28 19:50 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Oct 28 19:50 time.db
-rw-r--r--. 1 dune009 dune        519 Oct 28 18:53 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Oct 28 16:52 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Oct 28 19:50 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Oct 28 16:35 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Oct 28 19:50 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Oct 28 19:00 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Oct 28 18:53 debugprod.log
justIN time: 2025-11-05 17:37:49 UTC       justIN version: 01.05.01