justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 265177.1@dunegpschedd02.fnal.gov

Jobsub ID265177.1@dunegpschedd02.fnal.gov
Workflow ID11433
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-30 21:37:07
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce508
Last heartbeat2025-12-31 00:50:53
From worker nodeHostnameb9p12p9953.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-30 21:37:47
Input filesmonte-carlo-011433-000001
JobscriptExit code0
Real time3h (11505s)
CPU time2h (10590s = 92%)
Max RSS bytes4093128704 (3903 MiB)
Outputting started2025-12-31 00:49:33
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11433/1/001/reco_stage1.root
Finished2025-12-31 00:50:53
Saved logsjustin-logs:265177.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ol adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 127729 traces, 0 summary
Retagger: tagging trace set: gauss with 93935 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 93935 traces tagged "gauss"
FrameSaver: q=3.1328e+06 n=1608689 tag=gauss
wclsFrameSaver: saving 127729 traces tagged "wiener"
FrameSaver: q=3.33297e+06 n=1401520 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 38943 XUs and 21147 XVs -> 532 XUVs
C:0 T:1 81686 XUs and 53421 XVs -> 7520 XUVs
C:0 T:2 67556 XUs and 94773 XVs -> 4121 XUVs
C:0 T:3 38108 XUs and 38910 XVs -> 195 XUVs
C:0 T:4 12978 XUs and 15105 XVs -> 234 XUVs
C:0 T:5 62200 XUs and 72388 XVs -> 10381 XUVs
C:0 T:6 31141 XUs and 40810 XVs -> 1477 XUVs
C:0 T:7 12946 XUs and 17155 XVs -> 67 XUVs
C:0 T:8 11466 XUs and 11586 XVs -> 201 XUVs
C:0 T:9 77016 XUs and 96805 XVs -> 5250 XUVs
C:0 T:10 35005 XUs and 22861 XVs -> 4483 XUVs
C:0 T:11 68 XUs and 36 XVs -> 4 XUVs
34465 XUVs total
11757 collection wire objects
32050 potential space points
Neighbour search...
2351304 tests to find 1236758 neighbours
Iterating with no regularization...
Begin: 4.40971e+07
0 3.34433e+07
1 3.26878e+07
2 3.25734e+07
3 3.25397e+07
4 3.25249e+07
Now with regularization...
Begin: 2.32693e+07
0 2.32439e+07
1 2.32286e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411441846
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411441846
fLifetime: 30000 [us]
31-Dec-2025 01:49:31 CET  Closed output file "reco_stage1.root"
31-Dec-2025 01:49:31 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          254.579       294.152       352.066       286.764       33.8136        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000799057   0.00342638    0.00532166    0.00347362    0.00142536       10     
reco:ophit:OpHitFinder                             0.232022      0.259477      0.315661      0.250413      0.0259807       10     
reco:opflash:OpFlashFinder                         0.142119      0.167508      0.197318      0.164485      0.0182961       10     
reco:opslicer:OpSlicer                             0.501129      0.577064      0.774256      0.565107      0.0831823       10     
reco:caldata:DataPrepModule                         4.00308       4.77445       6.78715       4.35857      0.862623        10     
reco:wclsdatasp:WireCellToolkit                     27.2473       28.7129       30.9769       28.5402       1.18671        10     
reco:gaushit:GausHitFinder                         0.750873      0.988496       1.34421      0.980036      0.171425        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000337103   0.000436603   0.000558825   0.000430898   7.38248e-05      10     
reco:reco3d:SpacePointSolver                        9.7496        15.3667       26.092        13.0528       5.7454         10     
reco:hitpdune:DisambigFromSpacePoints               1.04416       1.50217       2.15906       1.43478      0.386376        10     
reco:pandora:StandardPandora                        19.8706       28.4553       41.1091       25.6305       7.34986        10     
reco:pandoraWriter:StandardPandora                  0.30051      0.358739      0.456139      0.352367      0.0506426       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.34896       3.05541       5.19958       2.98654      0.781217        10     
reco:pandoraShower:LArPandoraShowerCreation         2.15216       3.20152       6.31552       2.8954        1.14925        10     
reco:pandoracalo:Calorimetry                        1.39178       1.84506       3.21335       1.66814      0.512904        10     
reco:pandoracalonosce:Calorimetry                   1.29441       1.68089       2.44846       1.64217      0.326532        10     
reco:pandorapid:Chi2ParticleID                    0.00373061    0.00461569    0.00660207    0.00432042    0.000924848      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0355728     0.0425076     0.0673793     0.0396547    0.00898449       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00362928    0.00431623    0.00637664    0.00399495    0.000781101      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.45069       3.44787       5.07964       3.13312      0.740238        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.48086       3.53251       5.13247       3.50803      0.739614        10     
reco:emtrkmichelid:EmTrackMichelId                  140.027       160.388       186.804       156.478       17.0803        10     
reco:crttag:SingleCRTMatchingProducer             0.00846802     0.0141683     0.0175405     0.0142422    0.00238705       10     
reco:crtreco:TwoCRTMatchingProducer               0.00564233    0.00879516     0.0105772    0.00945084    0.00167813       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.616062      0.894001       1.15972      0.935849       0.17503        10     
reco:pandora2Track:LArPandoraTrackCreation          7.36115       9.80024       13.3824       9.78239       1.63886        10     
reco:pandora2calo:Calorimetry                       3.76876       4.97188       7.01727       4.57186       1.12402        10     
reco:pandora2calonosce:Calorimetry                  3.60805       5.13485       6.95156       5.12901       1.04025        10     
reco:pandora2pid:Chi2ParticleID                   0.00795163     0.0130596     0.0190729     0.0138134    0.00332451       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0434839     0.0636068     0.0922037     0.0609287     0.015771        10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00771298     0.012294      0.0187493     0.0121108     0.0031922       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.61826       3.52984       4.80557       3.43031      0.712954        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.28916       4.50162       6.74474       4.39997       1.06686        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.06125       4.2465        6.68569       3.82842       1.10817        10     
[art]:TriggerResults:TriggerResultInserter         4.051e-05    6.30384e-05   0.000168731   4.6495e-05    3.68018e-05      10     
end_path:out1:RootOutput                           9.23e-06     1.1423e-05     1.641e-05    1.0465e-05    2.13983e-06      10     
end_path:out1:RootOutput(write)                     1.97349       2.48861       3.3435        2.33748      0.457871        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4949.39 MB
  Peak resident set size usage (VmHWM): 4093.13 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.02 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10116904
-rw-r--r--. 1 duneprd np-comp 2371373641 Dec 31 00:57 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2248726025 Dec 31 00:55 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2068841685 Dec 31 01:49 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1831016828 Dec 30 23:01 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1761980081 Dec 30 22:45 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   77159190 Dec 31 01:46 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242697 Dec 31 01:49 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Dec 31 01:49 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Dec 31 01:49 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Dec 31 01:49 time.db
-rw-r--r--. 1 duneprd np-comp        519 Dec 31 00:57 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Dec 30 23:01 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Dec 31 01:49 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Dec 30 22:37 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Dec 31 01:49 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Dec 31 01:03 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Dec 31 00:57 debugprod.log
justIN time: 2026-02-07 00:49:52 UTC       justIN version: 01.06.00