justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 293917.1@dunegpschedd02.fnal.gov

Jobsub ID293917.1@dunegpschedd02.fnal.gov
Workflow ID12280
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-23 00:53:39
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce505
Last heartbeat2026-01-23 06:34:48
From worker nodeHostnameb9p13p1594.cern.ch
cpuinfoIntel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-23 00:55:09
Input filesmonte-carlo-012280-000001
JobscriptExit code0
Real time5h (20288s)
CPU time5h (19535s = 96%)
Max RSS bytes4416974848 (4212 MiB)
Outputting started2026-01-23 06:33:18
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12280/1/001/reco_stage1.root
Finished2026-01-23 06:34:48
Saved logsjustin-logs:293917.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 120161 traces, 0 summary
Retagger: tagging trace set: gauss with 95604 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 95604 traces tagged "gauss"
FrameSaver: q=3.34759e+06 n=1766588 tag=gauss
wclsFrameSaver: saving 120161 traces tagged "wiener"
FrameSaver: q=3.5251e+06 n=1596167 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 32536 XUs and 27246 XVs -> 453 XUVs
C:0 T:1 78951 XUs and 62178 XVs -> 5612 XUVs
C:0 T:2 61486 XUs and 105874 XVs -> 3613 XUVs
C:0 T:3 34802 XUs and 50426 XVs -> 267 XUVs
C:0 T:4 14240 XUs and 11975 XVs -> 176 XUVs
C:0 T:5 47367 XUs and 30966 XVs -> 4222 XUVs
C:0 T:6 110011 XUs and 131244 XVs -> 4280 XUVs
C:0 T:7 46249 XUs and 53506 XVs -> 434 XUVs
C:0 T:8 6590 XUs and 7231 XVs -> 51 XUVs
C:0 T:9 35770 XUs and 47443 XVs -> 2297 XUVs
C:0 T:10 46978 XUs and 36152 XVs -> 5678 XUVs
C:0 T:11 88 XUs and 85 XVs -> 10 XUVs
27093 XUVs total
10790 collection wire objects
24993 potential space points
Neighbour search...
1259167 tests to find 683818 neighbours
Iterating with no regularization...
Begin: 6.52111e+07
0 5.54834e+07
1 5.48563e+07
2 5.47836e+07
3 5.47668e+07
Now with regularization...
Begin: 4.60028e+07
0 4.60037e+07
Warning: metric increased
run: 1 ; subrun: 0 ; event: 10
evttime: 411905262
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411905262
fLifetime: 30000 [us]
23-Jan-2026 07:33:15 CET  Closed output file "reco_stage1.root"
23-Jan-2026 07:33:15 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          497.637       814.534       1048.72       850.988       165.728        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00161912    0.00464572     0.0088065    0.00468066    0.00201451       10     
reco:ophit:OpHitFinder                             0.421844      0.552951      0.694891       0.56064      0.0713084       10     
reco:opflash:OpFlashFinder                          0.20144      0.329909      0.461282      0.327208      0.068377        10     
reco:opslicer:OpSlicer                             0.915316       1.3279        1.79463       1.29468      0.259907        10     
reco:caldata:DataPrepModule                         10.4906       11.3576       12.1029       11.4181      0.455132        10     
reco:wclsdatasp:WireCellToolkit                     71.2419       77.4783       79.8616       78.8417       2.73623        10     
reco:gaushit:GausHitFinder                          1.81802       2.9825        3.69328       3.13498      0.551133        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000669128   0.00101556    0.00115406     0.0010457    0.000132246      10     
reco:reco3d:SpacePointSolver                        18.0149       29.7271       43.6249       28.5192       7.73962        10     
reco:hitpdune:DisambigFromSpacePoints               2.41904       4.80576       6.67186       4.78795       1.24352        10     
reco:pandora:StandardPandora                        37.0415       78.394        104.478       83.2416       22.2409        10     
reco:pandoraWriter:StandardPandora                 0.622695      0.902348       1.12232      0.924504      0.148501        10     
reco:pandoraTrack:LArPandoraTrackCreation           6.4021        13.5615       21.7293       14.5701       4.42517        10     
reco:pandoraShower:LArPandoraShowerCreation         4.47372       14.3386       22.2881       14.6042       5.03901        10     
reco:pandoracalo:Calorimetry                        3.03146       6.43012       8.44074       7.05088       1.85343        10     
reco:pandoracalonosce:Calorimetry                   2.57283       5.96017       9.26603       6.7405        1.95129        10     
reco:pandorapid:Chi2ParticleID                    0.00744401     0.0120612     0.0172825      0.01224     0.00267736       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0755685     0.114182      0.156779      0.123009      0.0271536       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00700882     0.0114072     0.0162086     0.0118734    0.00258066       10     
reco:pandoraShowercalo:ShowerCalorimetry            3.96097       14.7722       23.3482       15.3027       5.78029        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.39185       14.2838       22.6783       15.0489       5.65778        10     
reco:emtrkmichelid:EmTrackMichelId                  277.491       404.661       490.653       426.181       66.8993        10     
reco:crttag:SingleCRTMatchingProducer              0.0323875     0.0409483     0.0528786     0.0397473    0.00532662       10     
reco:crtreco:TwoCRTMatchingProducer                0.0201359     0.0296411     0.0391448     0.029899     0.00609752       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.05765       1.9846        2.86672       2.09659      0.478213        10     
reco:pandora2Track:LArPandoraTrackCreation          18.1882       44.4177       66.9002       48.6712       14.2011        10     
reco:pandora2calo:Calorimetry                       7.5971         19.22        31.7692       20.8908       6.87468        10     
reco:pandora2calonosce:Calorimetry                  5.70086       18.108        30.9096       19.1966       7.02517        10     
reco:pandora2pid:Chi2ParticleID                    0.0187548     0.0271624     0.0403168     0.0290785     0.0066505       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0918826     0.136663      0.177789       0.13977      0.0303004       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0187629     0.0280739     0.0380207     0.0300829    0.00585505       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.28952       13.1322       18.9166       13.9738       4.15685        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.15989       16.0926       23.3229       17.1219       5.53416        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.47203       14.079        19.5572       15.3694       4.7355         10     
[art]:TriggerResults:TriggerResultInserter        8.5178e-05    9.61465e-05   0.000132902   9.3889e-05    1.31177e-05      10     
end_path:out1:RootOutput                           8.39e-06     1.15521e-05   1.3768e-05    1.11705e-05   1.65952e-06      10     
end_path:out1:RootOutput(write)                     2.62169       5.04097       6.19679       5.35745       1.05794        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5237.22 MB
  Peak resident set size usage (VmHWM): 4416.97 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.44 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   7.18 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.42 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10687508
-rw-r--r--. 1 duneprd np-comp 2502326035 Jan 23 05:12 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2379749711 Jan 23 05:08 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2189005051 Jan 23 07:33 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1938621088 Jan 23 02:31 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1857943989 Jan 23 02:08 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   75745250 Jan 23 07:22 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242699 Jan 23 07:33 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 23 07:33 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 23 07:33 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 23 07:33 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 23 05:12 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 23 02:31 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 23 07:33 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 23 01:55 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 23 07:33 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 23 05:22 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 23 05:12 debugprod.log
justIN time: 2026-02-05 07:07:24 UTC       justIN version: 01.06.00