justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 294420.1@dunegpschedd02.fnal.gov

Jobsub ID294420.1@dunegpschedd02.fnal.gov
Workflow ID12345
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-24 00:20:56
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce515
Last heartbeat2026-01-24 04:19:01
From worker nodeHostnameb9p11p4111.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-24 00:22:54
Input filesmonte-carlo-012345-000001
JobscriptExit code0
Real time3h (14062s)
CPU time3h (13355s = 94%)
Max RSS bytes4405465088 (4201 MiB)
Outputting started2026-01-24 04:17:16
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12345/1/001/reco_stage1.root
Finished2026-01-24 04:19:01
Saved logsjustin-logs:294420.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ce:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 156624 traces, 0 summary
Retagger: tagging trace set: gauss with 119297 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 119297 traces tagged "gauss"
FrameSaver: q=4.00669e+06 n=2101227 tag=gauss
wclsFrameSaver: saving 156624 traces tagged "wiener"
FrameSaver: q=4.24176e+06 n=1853065 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 34547 XUs and 37625 XVs -> 491 XUVs
C:0 T:1 83608 XUs and 88214 XVs -> 11883 XUVs
C:0 T:2 79452 XUs and 94916 XVs -> 4632 XUVs
C:0 T:3 38113 XUs and 42189 XVs -> 183 XUVs
C:0 T:4 9892 XUs and 13836 XVs -> 63 XUVs
C:0 T:5 41956 XUs and 48840 XVs -> 4958 XUVs
C:0 T:6 161560 XUs and 89622 XVs -> 6454 XUVs
C:0 T:7 57754 XUs and 29171 XVs -> 368 XUVs
C:0 T:8 9491 XUs and 13953 XVs -> 107 XUVs
C:0 T:9 71971 XUs and 122921 XVs -> 8128 XUVs
C:0 T:10 90302 XUs and 37811 XVs -> 6388 XUVs
C:0 T:11 100 XUs and 71 XVs -> 6 XUVs
43661 XUVs total
14974 collection wire objects
39534 potential space points
Neighbour search...
4454934 tests to find 2106816 neighbours
Iterating with no regularization...
Begin: 8.57363e+07
0 7.22084e+07
1 7.12932e+07
2 7.11712e+07
3 7.11396e+07
Now with regularization...
Begin: 5.96923e+07
0 5.96471e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411927115
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411927115
fLifetime: 30000 [us]
24-Jan-2026 05:17:13 CET  Closed output file "reco_stage1.root"
24-Jan-2026 05:17:13 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          242.232       370.279       500.642       383.057       75.0972        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00104656     0.0037482    0.00616411    0.00386197    0.00170252       10     
reco:ophit:OpHitFinder                             0.224568      0.311773       0.42318      0.294672      0.0626734       10     
reco:opflash:OpFlashFinder                         0.150936      0.192924      0.234056       0.19175      0.0313576       10     
reco:opslicer:OpSlicer                             0.568162      0.731701       1.0263       0.675005      0.152773        10     
reco:caldata:DataPrepModule                         4.27737       5.84542       7.1052        5.87733      0.949089        10     
reco:wclsdatasp:WireCellToolkit                     28.1605       33.1488       36.2039       34.5714       2.96562        10     
reco:gaushit:GausHitFinder                          1.00237       1.32255       1.68231       1.26116      0.252243        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000347873   0.000509929   0.000735567   0.000492945   0.000115474      10     
reco:reco3d:SpacePointSolver                        8.11644       16.9103       27.7672       14.8637       6.87488        10     
reco:hitpdune:DisambigFromSpacePoints                1.038        1.98384       3.46901       2.06895      0.796839        10     
reco:pandora:StandardPandora                        12.7172       36.4355       53.0626       36.5421       10.8278        10     
reco:pandoraWriter:StandardPandora                 0.207024      0.420317      0.590012      0.421656      0.0979723       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.50105       3.28494       4.73899       3.33467      0.923768        10     
reco:pandoraShower:LArPandoraShowerCreation         1.39401       3.95683       8.06923       3.71551       1.69104        10     
reco:pandoracalo:Calorimetry                        1.31862       2.16147       3.27346       1.99615      0.632807        10     
reco:pandoracalonosce:Calorimetry                   1.16954       2.08165       2.9976        2.03346      0.559148        10     
reco:pandorapid:Chi2ParticleID                    0.00322276    0.00647629    0.00847781    0.00731698     0.0016405       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0345419     0.0636444     0.0874674     0.0674441     0.0180952       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00308404    0.00595529    0.00782567    0.00617781     0.0014944       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.83405       4.88772       8.26434       5.11907       1.80906        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.76531       5.05361       8.97785       5.47937        1.965         10     
reco:emtrkmichelid:EmTrackMichelId                  145.472       202.454       260.684       208.861       36.3827        10     
reco:crttag:SingleCRTMatchingProducer              0.0106184     0.0164348     0.0232963     0.014336     0.00428463       10     
reco:crtreco:TwoCRTMatchingProducer               0.00675404    0.00924858     0.0120905    0.00990917    0.00190766       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.705402       1.01259       1.2695        1.02986      0.176899        10     
reco:pandora2Track:LArPandoraTrackCreation          6.10445       14.0678       25.1189       15.1784       5.15326        10     
reco:pandora2calo:Calorimetry                       2.97248       7.26139       12.2973       7.64575       2.58824        10     
reco:pandora2calonosce:Calorimetry                  2.77742       6.72697       11.6338       7.28756       2.52337        10     
reco:pandora2pid:Chi2ParticleID                   0.00919929     0.0152758     0.0236352     0.0140055    0.00478294       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0456401     0.0775064     0.114803      0.0712672     0.0230148       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00899837     0.0151575     0.0227327     0.0139965    0.00474567       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.3268        4.89497       8.08981       4.49292       1.71025        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.18267       6.12926       9.48928       6.10363       2.13094        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.66522       5.48541       7.95383       5.62154       1.92089        10     
[art]:TriggerResults:TriggerResultInserter         4.41e-05     7.05965e-05   0.000138791   6.03355e-05   3.01808e-05      10     
end_path:out1:RootOutput                           8.61e-06     1.1079e-05     1.864e-05      9.9e-06     2.74768e-06      10     
end_path:out1:RootOutput(write)                     1.80619       3.20699       4.61198       3.09395      0.706262        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5234.01 MB
  Peak resident set size usage (VmHWM): 4405.47 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.27 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.70 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10872616
-rw-r--r--. 1 duneprd np-comp 2531036784 Jan 24 04:12 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2400957706 Jan 24 04:10 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2248138389 Jan 24 05:17 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1977563406 Jan 24 01:45 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1893157067 Jan 24 01:32 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   82093130 Jan 24 05:11 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242817 Jan 24 05:17 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 24 05:17 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 24 05:17 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 24 05:17 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 24 04:12 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 24 01:45 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 24 05:17 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 24 01:22 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 24 05:17 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 24 04:19 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 24 04:12 debugprod.log
justIN time: 2026-02-05 03:00:34 UTC       justIN version: 01.06.00