justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 238434.1@dunegpschedd01.fnal.gov

Jobsub ID238434.1@dunegpschedd01.fnal.gov
Workflow ID9329
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-25 07:27:02
SiteUK_Sheffield
EntryDUNE_UK_Sheffield_lcgce2
Last heartbeat2025-10-25 11:18:20
From worker nodeHostnamewn032.hep
cpuinfoIntel(R) Xeon(R) Gold 6226 CPU @ 2.70GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-25 07:28:21
Input filesmonte-carlo-009329-000001
JobscriptExit code0
Real time3h (13717s)
CPU time3h (13351s = 97%)
Max RSS bytes4212609024 (4017 MiB)
Outputting started2025-10-25 11:16:59
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09329/1/001/reco_stage1.root
Finished2025-10-25 11:18:20
Saved logsjustin-logs:238434.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

gnalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 94467 traces, 0 summary
Retagger: tagging trace set: gauss with 67061 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 67061 traces tagged "gauss"
FrameSaver: q=2.32896e+06 n=1133045 tag=gauss
wclsFrameSaver: saving 94467 traces tagged "wiener"
FrameSaver: q=2.48429e+06 n=977663 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 12855 XUs and 18473 XVs -> 175 XUVs
C:0 T:1 22475 XUs and 28212 XVs -> 2574 XUVs
C:0 T:2 50748 XUs and 85281 XVs -> 5068 XUVs
C:0 T:3 26632 XUs and 33615 XVs -> 208 XUVs
C:0 T:4 6098 XUs and 11618 XVs -> 58 XUVs
C:0 T:5 31686 XUs and 53058 XVs -> 10477 XUVs
C:0 T:6 52474 XUs and 65107 XVs -> 5624 XUVs
C:0 T:8 4550 XUs and 6758 XVs -> 44 XUVs
C:0 T:9 40591 XUs and 58311 XVs -> 4823 XUVs
C:0 T:10 20708 XUs and 27931 XVs -> 5253 XUVs
C:0 T:11 111 XUs and 63 XVs -> 13 XUVs
34317 XUVs total
8376 collection wire objects
32379 potential space points
Neighbour search...
4245797 tests to find 2147214 neighbours
Iterating with no regularization...
Begin: 3.09829e+07
0 2.47896e+07
1 2.41474e+07
2 2.40364e+07
3 2.40007e+07
4 2.39841e+07
Now with regularization...
Begin: 1.65934e+07
0 1.65518e+07
1 1.65229e+07
2 1.65011e+07
3 1.64835e+07
4 1.6469e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410102370
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410102370
fLifetime: 30000 [us]
25-Oct-2025 12:16:56 BST  Closed output file "reco_stage1.root"
25-Oct-2025 12:16:56 BST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          346.206       568.808       989.101       544.183       164.481        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00122812    0.00374114    0.00676398    0.00392046    0.00151233       10     
reco:ophit:OpHitFinder                             0.205887      0.388993      0.463059      0.401546      0.0681188       10     
reco:opflash:OpFlashFinder                         0.116897      0.231166      0.327941      0.229922      0.0576623       10     
reco:opslicer:OpSlicer                             0.434869      0.935943       1.15773      0.945156      0.192742        10     
reco:caldata:DataPrepModule                         6.05523       8.70207       9.42133        8.983       0.908966        10     
reco:wclsdatasp:WireCellToolkit                     49.1173       59.7101       68.463        60.7951       6.38992        10     
reco:gaushit:GausHitFinder                         0.880267       1.74507       2.28681       1.71923      0.395211        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000500026   0.000752735    0.0009968    0.00072725    0.000157491      10     
reco:reco3d:SpacePointSolver                        9.63022       16.7949       23.9725       17.7764       5.53226        10     
reco:hitpdune:DisambigFromSpacePoints               1.47573       2.70543       4.79131       2.67005       1.10642        10     
reco:pandora:StandardPandora                        17.1903       48.2503       125.684       39.0822       27.7957        10     
reco:pandoraWriter:StandardPandora                 0.405121      0.640485      0.809748      0.675739      0.128935        10     
reco:pandoraTrack:LArPandoraTrackCreation           4.27478       9.69275       16.7184       9.67593       3.26735        10     
reco:pandoraShower:LArPandoraShowerCreation         3.23016       9.46482       23.4835       8.6111        5.34061        10     
reco:pandoracalo:Calorimetry                        2.06022       4.49395       6.74506       4.6732        1.28521        10     
reco:pandoracalonosce:Calorimetry                    1.798         4.288        6.73352       4.43006       1.31766        10     
reco:pandorapid:Chi2ParticleID                    0.00586623     0.0091451     0.0112974     0.0093548     0.0016047       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0640082     0.0913624     0.143315      0.0932151     0.0210523       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0058918    0.00870685     0.0109636    0.00890027    0.00132392       10     
reco:pandoraShowercalo:ShowerCalorimetry            3.10401       9.41098       24.1578       8.0868        5.73583        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.23301       9.20175       22.9428       7.89516       5.50471        10     
reco:emtrkmichelid:EmTrackMichelId                  205.229       297.514       483.416       290.09        71.4102        10     
reco:crttag:SingleCRTMatchingProducer              0.0151474     0.0311689     0.0458034     0.0311043    0.00807729       10     
reco:crtreco:TwoCRTMatchingProducer               0.00924937     0.0225994     0.0349214     0.0227374    0.00703556       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.883019       1.49216       2.6073        1.43595      0.442526        10     
reco:pandora2Track:LArPandoraTrackCreation          7.75603       28.5986       59.7709       26.9275       13.2527        10     
reco:pandora2calo:Calorimetry                       3.69519       12.6765       29.5793       10.7103       6.75721        10     
reco:pandora2calonosce:Calorimetry                  3.26032       11.6673       29.8072       10.2217       6.95401        10     
reco:pandora2pid:Chi2ParticleID                   0.00812582     0.018467      0.0353622     0.0180633    0.00732114       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0518268     0.0975129     0.197208      0.0815166     0.0393062       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00816598     0.017612      0.0344249     0.0151224    0.00719957       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.44943       8.02172       18.7021       6.6944        4.3737         10     
reco:pandora2Showercalo:ShowerCalorimetry           2.57698       9.56062       21.954         8.782        5.28606        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.33209       8.49062       18.5789       8.40352       4.46314        10     
[art]:TriggerResults:TriggerResultInserter        4.6681e-05    7.77061e-05   0.000168439   7.5248e-05    3.38338e-05      10     
end_path:out1:RootOutput                           7.114e-06    1.0979e-05    2.1722e-05    1.0016e-05    4.04389e-06      10     
end_path:out1:RootOutput(write)                     1.98185       3.69184       5.69788       3.68779      0.953059        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5021.9 MB
  Peak resident set size usage (VmHWM): 4212.61 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.34 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.51 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.31 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9796988
-rw-r--r--. 1 dune004 dune 2313542536 Oct 25 10:39 detsim_stage2.root
-rw-r--r--. 1 dune004 dune 2199533526 Oct 25 10:36 detsim_stage1.root
-rw-r--r--. 1 dune004 dune 1985849904 Oct 25 12:16 reco_stage1.root
-rw-r--r--. 1 dune004 dune 1759894092 Oct 25 08:52 g4_stage2.root
-rw-r--r--. 1 dune004 dune 1701475382 Oct 25 08:36 g4_stage1.root
-rw-r--r--. 1 dune004 dune   71206430 Oct 25 12:12 Pandora_Events.pndr
-rw-r--r--. 1 dune004 dune     242794 Oct 25 12:16 jobscript.log
-rw-r--r--. 1 dune004 dune     217088 Oct 25 12:16 mem.db
-rw-r--r--. 1 dune004 dune      62376 Oct 25 12:16 Pandora_Geometry.xml
-rw-r--r--. 1 dune004 dune      36864 Oct 25 12:16 time.db
-rw-r--r--. 1 dune004 dune        519 Oct 25 10:39 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune004 dune        519 Oct 25 08:52 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune004 dune        519 Oct 25 12:16 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune004 dune         52 Oct 25 08:28 all-input-dids.txt
-rw-r--r--. 1 dune004 dune          7 Oct 25 12:16 justin-processed-pfns.txt
-rw-r--r--. 1 dune004 dune          0 Oct 25 10:47 ProtoDUNE.log
-rw-r--r--. 1 dune004 dune          0 Oct 25 10:39 debugprod.log
justIN time: 2025-11-03 18:43:37 UTC       justIN version: 01.05.01