justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 236954.1@dunegpschedd01.fnal.gov

Jobsub ID236954.1@dunegpschedd01.fnal.gov
Workflow ID9222
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-20 23:10:23
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2025-10-21 02:35:02
From worker nodeHostnamecn101.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-20 23:11:04
Input filesmonte-carlo-009222-000001
JobscriptExit code0
Real time3h (12030s)
CPU time3h (11813s = 98%)
Max RSS bytes4331986944 (4131 MiB)
Outputting started2025-10-21 02:31:35
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09222/1/001/reco_stage1.root
Finished2025-10-21 02:35:02
Saved logsjustin-logs:236954.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 114471 traces, 0 summary
Retagger: tagging trace set: gauss with 89901 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 89901 traces tagged "gauss"
FrameSaver: q=3.06506e+06 n=1587990 tag=gauss
wclsFrameSaver: saving 114471 traces tagged "wiener"
FrameSaver: q=3.24826e+06 n=1417204 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 42581 XUs and 52967 XVs -> 643 XUVs
C:0 T:1 85896 XUs and 114153 XVs -> 6643 XUVs
C:0 T:2 71802 XUs and 95579 XVs -> 2954 XUVs
C:0 T:3 37905 XUs and 44364 XVs -> 144 XUVs
C:0 T:4 12664 XUs and 14264 XVs -> 202 XUVs
C:0 T:5 64910 XUs and 73416 XVs -> 8316 XUVs
C:0 T:6 63368 XUs and 56473 XVs -> 2078 XUVs
C:0 T:7 27710 XUs and 24491 XVs -> 191 XUVs
C:0 T:8 4319 XUs and 3002 XVs -> 21 XUVs
C:0 T:9 32287 XUs and 28593 XVs -> 2436 XUVs
C:0 T:10 29873 XUs and 17460 XVs -> 2897 XUVs
C:0 T:11 244 XUs and 189 XVs -> 36 XUVs
26561 XUVs total
10480 collection wire objects
24528 potential space points
Neighbour search...
1354748 tests to find 680608 neighbours
Iterating with no regularization...
Begin: 8.03735e+07
0 6.64114e+07
1 6.53476e+07
2 6.5201e+07
3 6.5156e+07
Now with regularization...
Begin: 5.61769e+07
0 5.61526e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410015128
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410015128
fLifetime: 30000 [us]
21-Oct-2025 03:31:32 BST  Closed output file "reco_stage1.root"
21-Oct-2025 03:31:32 BST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          344.682       524.759       767.223       536.915       135.916        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00104081    0.00492058     0.0112334    0.00395886    0.00325301       10     
reco:ophit:OpHitFinder                             0.230912      0.280959      0.364817      0.264215      0.0441729       10     
reco:opflash:OpFlashFinder                         0.176376      0.228311      0.283167      0.235198      0.0344018       10     
reco:opslicer:OpSlicer                             0.510081      0.667743      0.994542      0.635626      0.156003        10     
reco:caldata:DataPrepModule                         4.96354       6.14446       9.04355       5.25028       1.57565        10     
reco:wclsdatasp:WireCellToolkit                     44.9282       52.2279       67.4626       48.1049       8.28407        10     
reco:gaushit:GausHitFinder                          1.00508       1.41263       1.84909       1.47321      0.245782        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000659408   0.00088707    0.00110157     0.0009112    0.000128805      10     
reco:reco3d:SpacePointSolver                        10.1977       15.228        21.3087       15.0931       3.18688        10     
reco:hitpdune:DisambigFromSpacePoints               1.38246       2.34214       3.64963       2.25735      0.768214        10     
reco:pandora:StandardPandora                        22.2538       47.2211       69.4866       53.2226       16.2676        10     
reco:pandoraWriter:StandardPandora                 0.396662      0.561169      0.773629      0.525522      0.135348        10     
reco:pandoraTrack:LArPandoraTrackCreation           6.99436       13.532        22.6955       13.1149       4.16413        10     
reco:pandoraShower:LArPandoraShowerCreation         10.4389       15.7575       20.104        16.9608       3.65318        10     
reco:pandoracalo:Calorimetry                        3.05045       6.59473       10.2982       6.06529       2.22364        10     
reco:pandoracalonosce:Calorimetry                   2.84196       6.18538       9.70626       5.8732        2.02557        10     
reco:pandorapid:Chi2ParticleID                    0.00441451    0.00686603     0.0105493    0.00617334    0.00196437       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0539945     0.0700034     0.134125      0.0574536     0.0238835       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00436432    0.00685661     0.0134985    0.00564774    0.00254079       10     
reco:pandoraShowercalo:ShowerCalorimetry            7.58825       15.0309       22.5273       15.5193       4.82285        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       7.51661       14.6388       22.1775       15.2541       4.77763        10     
reco:emtrkmichelid:EmTrackMichelId                  137.784       199.341       330.249       176.375       62.6895        10     
reco:crttag:SingleCRTMatchingProducer              0.0250593     0.034635      0.0458052     0.0345064    0.00676425       10     
reco:crtreco:TwoCRTMatchingProducer                0.0211407     0.0285595     0.0370234     0.0284457    0.00562137       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.908829       1.16056       1.73406       1.03777      0.282902        10     
reco:pandora2Track:LArPandoraTrackCreation          25.2151       41.7762       61.4443       40.1083       11.7207        10     
reco:pandora2calo:Calorimetry                       9.45607       19.356        31.071        18.2788       6.65702        10     
reco:pandora2calonosce:Calorimetry                  9.62681       18.4437       29.4386       17.3612       6.27244        10     
reco:pandora2pid:Chi2ParticleID                   0.00995558     0.0164521     0.0296861     0.0148437    0.00620858       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0632493     0.0845129     0.133027      0.0740814     0.0236623       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0102587     0.0151132     0.0246487     0.0135233    0.00483472       10     
reco:pandora2Shower:LArPandoraShowerCreation        10.0114       13.2001       18.7802       12.167        2.97291        10     
reco:pandora2Showercalo:ShowerCalorimetry           8.76292       16.1802       26.9293       14.6596       5.63776        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      7.48829       13.9603       23.6617       13.0237       4.83347        10     
[art]:TriggerResults:TriggerResultInserter        4.7526e-05    6.46848e-05   0.000118815   5.0839e-05    2.37444e-05      10     
end_path:out1:RootOutput                           6.749e-06     8.83e-06      1.216e-05    8.2825e-06    1.6788e-06       10     
end_path:out1:RootOutput(write)                     2.06552       2.81217       4.91358       2.45359      0.902914        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4996.78 MB
  Peak resident set size usage (VmHWM): 4192.74 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.85 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.20 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10404820
-rw-r--r-- 1 pildune32 pildune 2437053973 Oct 21 02:01 detsim_stage2.root
-rw-r--r-- 1 pildune32 pildune 2311744609 Oct 21 01:59 detsim_stage1.root
-rw-r--r-- 1 pildune32 pildune 2116640241 Oct 21 03:31 reco_stage1.root
-rw-r--r-- 1 pildune32 pildune 1897126375 Oct 21 00:36 g4_stage2.root
-rw-r--r-- 1 pildune32 pildune 1817824650 Oct 21 00:17 g4_stage1.root
-rw-r--r-- 1 pildune32 pildune   73535130 Oct 21 03:25 Pandora_Events.pndr
-rw-r--r-- 1 pildune32 pildune     243148 Oct 21 03:31 jobscript.log
-rw-r--r-- 1 pildune32 pildune     217088 Oct 21 03:31 mem.db
-rw-r--r-- 1 pildune32 pildune      62376 Oct 21 03:31 Pandora_Geometry.xml
-rw-r--r-- 1 pildune32 pildune      36864 Oct 21 03:31 time.db
-rw-r--r-- 1 pildune32 pildune        519 Oct 21 02:01 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Oct 21 00:36 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Oct 21 03:31 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune32 pildune         52 Oct 21 00:11 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune          7 Oct 21 03:31 justin-processed-pfns.txt
-rw-r--r-- 1 pildune32 pildune          0 Oct 21 02:07 ProtoDUNE.log
-rw-r--r-- 1 pildune32 pildune          0 Oct 21 02:01 debugprod.log
justIN time: 2025-11-04 07:43:22 UTC       justIN version: 01.05.01