justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 271136.0@dunegpschedd01.fnal.gov

Jobsub ID271136.0@dunegpschedd01.fnal.gov
Workflow ID11447
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-31 23:20:29
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce01
Last heartbeat2026-01-01 03:16:26
From worker nodeHostnameccwcondor0634
cpuinfoAMD EPYC 7453 28-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-31 23:21:00
Input filesmonte-carlo-011447-000001
JobscriptExit code0
Real time3h (14018s)
CPU time3h (13417s = 95%)
Max RSS bytes4067942400 (3879 MiB)
Outputting started2026-01-01 03:14:39
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11447/1/001/reco_stage1.root
Finished2026-01-01 03:16:26
Saved logsjustin-logs:271136.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

epare:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 124134 traces, 0 summary
Retagger: tagging trace set: gauss with 93396 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 93396 traces tagged "gauss"
FrameSaver: q=3.39071e+06 n=1641377 tag=gauss
wclsFrameSaver: saving 124134 traces tagged "wiener"
FrameSaver: q=3.61733e+06 n=1458236 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 35453 XUs and 42812 XVs -> 662 XUVs
C:0 T:1 73060 XUs and 81536 XVs -> 8844 XUVs
C:0 T:2 124901 XUs and 147821 XVs -> 6134 XUVs
C:0 T:3 63176 XUs and 61167 XVs -> 493 XUVs
C:0 T:4 9569 XUs and 12362 XVs -> 116 XUVs
C:0 T:5 40258 XUs and 50276 XVs -> 5123 XUVs
C:0 T:6 39214 XUs and 54032 XVs -> 3602 XUVs
C:0 T:7 16067 XUs and 21328 XVs -> 234 XUVs
C:0 T:8 7752 XUs and 6864 XVs -> 62 XUVs
C:0 T:9 90373 XUs and 61056 XVs -> 5642 XUVs
C:0 T:10 49372 XUs and 16879 XVs -> 7542 XUVs
C:0 T:11 292 XUs and 229 XVs -> 45 XUVs
38499 XUVs total
10528 collection wire objects
35471 potential space points
Neighbour search...
3301347 tests to find 1761320 neighbours
Iterating with no regularization...
Begin: 7.38402e+07
0 6.15202e+07
1 6.07359e+07
2 6.06198e+07
3 6.05827e+07
Now with regularization...
Begin: 5.03334e+07
0 5.03039e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411463674
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411463674
fLifetime: 30000 [us]
01-Jan-2026 04:14:36 CET  Closed output file "reco_stage1.root"
01-Jan-2026 04:14:36 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          344.235       474.834       603.284       491.83        73.6226        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00168285    0.00515394    0.00870733    0.00514343    0.00235204       10     
reco:ophit:OpHitFinder                             0.274184      0.347211      0.400239      0.359648      0.0412646       10     
reco:opflash:OpFlashFinder                         0.125885       0.19853      0.268924      0.197739      0.0409101       10     
reco:opslicer:OpSlicer                              0.48694      0.770362      0.933482       0.80131      0.130087        10     
reco:caldata:DataPrepModule                         6.5738        7.09841       7.72286       7.10503      0.350698        10     
reco:wclsdatasp:WireCellToolkit                     43.256        45.9287       49.4054       45.885        2.05288        10     
reco:gaushit:GausHitFinder                          1.27722       1.61957       2.28498       1.55103      0.301796        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00055518    0.00091288    0.00130444    0.000865991   0.000234632      10     
reco:reco3d:SpacePointSolver                        8.81797       19.4361       35.2998       18.7623       7.19327        10     
reco:hitpdune:DisambigFromSpacePoints               1.28577       1.91551       2.72466       2.00818      0.387489        10     
reco:pandora:StandardPandora                        26.4371       43.1155       84.7887       39.7824       15.2225        10     
reco:pandoraWriter:StandardPandora                 0.438057      0.572742      0.715286      0.559283      0.0871604       10     
reco:pandoraTrack:LArPandoraTrackCreation           3.98506       7.96496       11.4534       8.59354       2.15889        10     
reco:pandoraShower:LArPandoraShowerCreation         5.21778       9.8377        11.9383       10.4998       2.30632        10     
reco:pandoracalo:Calorimetry                        2.54258       5.06422       7.12938       5.35124       1.30919        10     
reco:pandoracalonosce:Calorimetry                   2.26466       4.72483       7.72135       4.9871        1.41356        10     
reco:pandorapid:Chi2ParticleID                    0.00590943    0.00783233    0.00920344    0.00826458     0.001112        10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0610734     0.080812      0.105417      0.0781851     0.0137699       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00580462    0.00730448     0.0106021    0.00705572    0.00135936       10     
reco:pandoraShowercalo:ShowerCalorimetry            5.79526       10.6834       15.5131       11.026        2.79678        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       5.70182       10.531        13.795        11.0959       2.52915        10     
reco:emtrkmichelid:EmTrackMichelId                  174.391       215.688       240.892       220.068       22.3484        10     
reco:crttag:SingleCRTMatchingProducer              0.0196508     0.0301138     0.0440809     0.0301434    0.00657694       10     
reco:crtreco:TwoCRTMatchingProducer                0.0174901     0.0234392     0.0310685     0.0225428    0.00361424       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.632514       1.01059       1.43553       1.01127      0.243465        10     
reco:pandora2Track:LArPandoraTrackCreation          13.3646       24.1867       31.1124       26.4171       5.57178        10     
reco:pandora2calo:Calorimetry                       6.8971        14.2831       19.2748       15.7157       3.84745        10     
reco:pandora2calonosce:Calorimetry                  5.87804       13.6932       18.9032       14.6885       4.03529        10     
reco:pandora2pid:Chi2ParticleID                    0.0143717     0.0173611     0.0242193     0.0168779    0.00285244       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0779813     0.0972461      0.12454      0.0985213     0.0131367       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0143564     0.0172575     0.0225438     0.0166483    0.00272495       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.82978       9.39726       12.0708       9.67865       2.2246         10     
reco:pandora2Showercalo:ShowerCalorimetry           4.88846       12.3588       16.7306       13.6431       3.38098        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.35532       10.7433       14.3347       11.5115       3.0624         10     
[art]:TriggerResults:TriggerResultInserter         8.007e-05    0.000115202   0.000219711   0.000101981   3.97379e-05      10     
end_path:out1:RootOutput                          1.2153e-05    2.0888e-05    4.7889e-05    1.7989e-05    9.67024e-06      10     
end_path:out1:RootOutput(write)                     2.51653       3.13852       3.87385       3.18121      0.383267        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4891.48 MB
  Peak resident set size usage (VmHWM): 4067.94 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.35 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.45 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.19 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9401432
-rw-r--r-- 1 dune001 lbno 2235144408 Jan  1 02:53 detsim_stage2.root
-rw-r--r-- 1 dune001 lbno 2121512599 Jan  1 02:50 detsim_stage1.root
-rw-r--r-- 1 dune001 lbno 1897498412 Jan  1 04:14 reco_stage1.root
-rw-r--r-- 1 dune001 lbno 1688455283 Jan  1 00:42 g4_stage2.root
-rw-r--r-- 1 dune001 lbno 1614825080 Jan  1 00:28 g4_stage1.root
-rw-r--r-- 1 dune001 lbno   69013800 Jan  1 04:08 Pandora_Events.pndr
-rw-r--r-- 1 dune001 lbno     242689 Jan  1 04:14 jobscript.log
-rw-r--r-- 1 dune001 lbno     217088 Jan  1 04:14 mem.db
-rw-r--r-- 1 dune001 lbno      62376 Jan  1 04:14 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 lbno      36864 Jan  1 04:14 time.db
-rw-r--r-- 1 dune001 lbno        519 Jan  1 02:53 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan  1 00:42 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan  1 04:14 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 lbno         52 Jan  1 00:21 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno          7 Jan  1 04:14 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno          0 Jan  1 02:59 ProtoDUNE.log
-rw-r--r-- 1 dune001 lbno          0 Jan  1 02:53 debugprod.log
justIN time: 2026-02-06 23:33:09 UTC       justIN version: 01.06.00