justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 271485.1@dunegpschedd02.fnal.gov

Jobsub ID271485.1@dunegpschedd02.fnal.gov
Workflow ID11833
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-13 15:41:30
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc02
Last heartbeat2026-01-13 20:22:40
From worker nodeHostnamewn-lb-06.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-13 15:42:26
Input filesmonte-carlo-011833-000001
JobscriptExit code0
Real time4h (16717s)
CPU time4h (16532s = 98%)
Max RSS bytes4207337472 (4012 MiB)
Outputting started2026-01-13 20:21:04
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11833/1/001/reco_stage1.root
Finished2026-01-13 20:22:34
Saved logsjustin-logs:271485.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 152600 traces, 0 summary
Retagger: tagging trace set: gauss with 121936 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 121936 traces tagged "gauss"
FrameSaver: q=4.28788e+06 n=2217294 tag=gauss
wclsFrameSaver: saving 152600 traces tagged "wiener"
FrameSaver: q=4.49257e+06 n=1994999 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 42348 XUs and 37390 XVs -> 330 XUVs
C:0 T:1 68884 XUs and 64035 XVs -> 3431 XUVs
C:0 T:2 128384 XUs and 191689 XVs -> 6793 XUVs
C:0 T:3 63161 XUs and 79012 XVs -> 688 XUVs
C:0 T:4 13735 XUs and 12993 XVs -> 102 XUVs
C:0 T:5 51372 XUs and 41855 XVs -> 2975 XUVs
C:0 T:6 140339 XUs and 136007 XVs -> 4886 XUVs
C:0 T:7 57441 XUs and 55159 XVs -> 484 XUVs
C:0 T:8 8714 XUs and 8763 XVs -> 89 XUVs
C:0 T:9 49282 XUs and 60049 XVs -> 2856 XUVs
C:0 T:10 28236 XUs and 22979 XVs -> 6504 XUVs
C:0 T:11 107 XUs and 63 XVs -> 4 XUVs
29142 XUVs total
11960 collection wire objects
26833 potential space points
Neighbour search...
1129123 tests to find 569018 neighbours
Iterating with no regularization...
Begin: 8.10506e+07
0 7.02398e+07
1 6.9509e+07
2 6.94241e+07
3 6.94027e+07
Now with regularization...
Begin: 5.85021e+07
0 5.85095e+07
Warning: metric increased
run: 1 ; subrun: 0 ; event: 10
evttime: 411718703
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411718703
fLifetime: 30000 [us]
13-Jan-2026 21:21:02 CET  Closed output file "reco_stage1.root"
13-Jan-2026 21:21:02 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          330.605       481.29        641.89        443.153       100.019        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00122533    0.00627287     0.0110578    0.00629094    0.00309709       10     
reco:ophit:OpHitFinder                             0.370497      0.436201      0.511886      0.430053      0.0444949       10     
reco:opflash:OpFlashFinder                         0.236332      0.318455      0.444136      0.310443      0.0549586       10     
reco:opslicer:OpSlicer                             0.645844      0.820192      0.961143      0.836291      0.101837        10     
reco:caldata:DataPrepModule                         6.32715       6.55634       7.32716       6.48665      0.271124        10     
reco:wclsdatasp:WireCellToolkit                     44.9255       66.3975       74.1094       71.2332       10.5936        10     
reco:gaushit:GausHitFinder                          1.03093       1.41806        1.822        1.4541       0.239709        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000404774   0.00059873    0.00122201    0.000548884   0.000228276      10     
reco:reco3d:SpacePointSolver                        14.2988       21.9188       34.3649       19.2687       6.85154        10     
reco:hitpdune:DisambigFromSpacePoints               1.04249       1.87197       2.89215       1.65582      0.645605        10     
reco:pandora:StandardPandora                        22.1375       51.0599       99.6488       54.4784       21.8295        10     
reco:pandoraWriter:StandardPandora                 0.359403      0.518137      0.645965      0.508065      0.0884068       10     
reco:pandoraTrack:LArPandoraTrackCreation           4.1466        7.87232       12.3913       8.38911       2.37901        10     
reco:pandoraShower:LArPandoraShowerCreation         2.72935       8.64281       14.1658       7.85688       3.46441        10     
reco:pandoracalo:Calorimetry                        2.14296       4.27725       6.55142       4.48024       1.20799        10     
reco:pandoracalonosce:Calorimetry                   1.99801       4.10392       6.23997       4.37433       1.19876        10     
reco:pandorapid:Chi2ParticleID                     0.0057678    0.00792697     0.0149286    0.00697338    0.00269945       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0432372     0.0571532     0.0695481     0.0566074    0.00839387       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00540679    0.00675824     0.0083435    0.00694461    0.00091952       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.45928       8.48818       14.2364       7.81686       3.50245        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.39917       8.06012       12.6014       7.52807       3.02575        10     
reco:emtrkmichelid:EmTrackMichelId                  164.255       217.331       271.94        199.918       37.8491        10     
reco:crttag:SingleCRTMatchingProducer              0.0149417     0.0234879     0.0367507     0.0242194    0.00656758       10     
reco:crtreco:TwoCRTMatchingProducer                0.0085108     0.0151171     0.0229147     0.015517     0.00393132       10     
reco:anodepiercerst0:T0RecoAnodePiercers            0.65815       1.07134       1.38233       1.07978      0.204948        10     
reco:pandora2Track:LArPandoraTrackCreation           8.502        21.6876       34.5181       18.3935       8.07712        10     
reco:pandora2calo:Calorimetry                       3.99852       10.9952       16.915        9.49104       4.41462        10     
reco:pandora2calonosce:Calorimetry                  3.79264       10.2888       15.6101       8.49757       4.03555        10     
reco:pandora2pid:Chi2ParticleID                    0.0111364     0.0150979     0.0196019     0.0141248    0.00294546       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0528979     0.0724693     0.0884254     0.0702426     0.0115169       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0110706     0.015099      0.0194033     0.0143529    0.00303006       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.10969       7.41822       10.8073       6.79943       2.42862        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.29176       8.65263       13.4726       7.68699        3.01          10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.88194       7.13239       10.5413       6.38535       2.31778        10     
[art]:TriggerResults:TriggerResultInserter        3.9187e-05    5.1835e-05    0.000136343   4.2458e-05    2.82807e-05      10     
end_path:out1:RootOutput                           7.601e-06    5.05405e-05   0.000432004   8.1125e-06    0.000127155      10     
end_path:out1:RootOutput(write)                     2.55069       3.4798        4.22389       3.43941      0.567985        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5134.03 MB
  Peak resident set size usage (VmHWM): 4207.34 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.29 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.06 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10870292
-rw-r--r--. 1 dune009 dune 2531710186 Jan 13 19:58 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2403812109 Jan 13 19:55 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2223401899 Jan 13 21:21 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1998576913 Jan 13 17:06 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1896691975 Jan 13 16:50 g4_stage1.root
-rw-r--r--. 1 dune009 dune   76372250 Jan 13 21:13 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     242989 Jan 13 21:21 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Jan 13 21:21 mem.db
-rw-r--r--. 1 dune009 dune      62376 Jan 13 21:21 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Jan 13 21:21 time.db
-rw-r--r--. 1 dune009 dune        519 Jan 13 19:58 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Jan 13 17:06 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Jan 13 21:21 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Jan 13 16:42 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Jan 13 21:21 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Jan 13 20:05 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Jan 13 19:58 debugprod.log
justIN time: 2026-02-06 05:56:12 UTC       justIN version: 01.06.00