justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 248815.0@dunegpschedd02.fnal.gov

Jobsub ID248815.0@dunegpschedd02.fnal.gov
Workflow ID10395
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-20 10:00:58
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2025-11-20 13:16:58
From worker nodeHostnamecn507.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-20 10:01:31
Input filesmonte-carlo-010395-000001
JobscriptExit code0
Real time3h (11519s)
CPU time3h (11350s = 98%)
Max RSS bytes3959140352 (3775 MiB)
Outputting started2025-11-20 13:13:31
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10395/1/001/reco_stage1.root
Finished2025-11-20 13:16:58
Saved logsjustin-logs:248815.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

pService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 128881 traces, 0 summary
Retagger: tagging trace set: gauss with 94260 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 94260 traces tagged "gauss"
FrameSaver: q=3.40542e+06 n=1621662 tag=gauss
wclsFrameSaver: saving 128881 traces tagged "wiener"
FrameSaver: q=3.62523e+06 n=1417842 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 21157 XUs and 27863 XVs -> 242 XUVs
C:0 T:1 46426 XUs and 42435 XVs -> 3277 XUVs
C:0 T:2 85957 XUs and 104674 XVs -> 5447 XUVs
C:0 T:3 43540 XUs and 45442 XVs -> 255 XUVs
C:0 T:4 8546 XUs and 11493 XVs -> 74 XUVs
C:0 T:5 32217 XUs and 42133 XVs -> 4684 XUVs
C:0 T:6 206820 XUs and 200679 XVs -> 9074 XUVs
C:0 T:7 77773 XUs and 67317 XVs -> 1089 XUVs
C:0 T:8 3630 XUs and 5708 XVs -> 233 XUVs
C:0 T:9 41499 XUs and 54102 XVs -> 8923 XUVs
C:0 T:10 21844 XUs and 10154 XVs -> 2374 XUVs
C:0 T:11 64 XUs and 26 XVs -> 5 XUVs
35677 XUVs total
10624 collection wire objects
31295 potential space points
Neighbour search...
2151675 tests to find 1120954 neighbours
Iterating with no regularization...
Begin: 5.58487e+07
0 3.99029e+07
1 3.86162e+07
2 3.83988e+07
3 3.83337e+07
4 3.83034e+07
Now with regularization...
Begin: 2.87993e+07
0 2.87075e+07
1 2.86795e+07
TestBeamParticleCreationAlgorithm::SetupTestBeamVertex - Test beam particle has no initial vertex
run: 1 ; subrun: 0 ; event: 10
evttime: 410627698
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410627698
fLifetime: 30000 [us]
20-Nov-2025 13:13:28 GMT  Closed output file "reco_stage1.root"
20-Nov-2025 13:13:28 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          330.199       478.146       615.067       476.744       79.0681        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                             0.0015371    0.00334971    0.00563204    0.00332734    0.00119308       10     
reco:ophit:OpHitFinder                             0.268647      0.334329      0.433376      0.313449      0.0593187       10     
reco:opflash:OpFlashFinder                         0.159406      0.204028      0.257264      0.206248      0.032197        10     
reco:opslicer:OpSlicer                             0.604874      0.811865       1.12785      0.834424       0.15335        10     
reco:caldata:DataPrepModule                         6.49186       7.63738       10.2334       6.72726       1.49955        10     
reco:wclsdatasp:WireCellToolkit                     52.3173       56.4474       62.0323       56.533        2.79866        10     
reco:gaushit:GausHitFinder                          1.0624        1.53408       2.03394       1.54943      0.249381        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000575809   0.000765971   0.000961137   0.000769505   0.00011529       10     
reco:reco3d:SpacePointSolver                        8.3936        15.9823       28.7479       12.9135       6.5906         10     
reco:hitpdune:DisambigFromSpacePoints               1.30344       2.2424        3.04435       2.32106      0.563102        10     
reco:pandora:StandardPandora                        17.2592       38.297        60.4107       38.8696       11.7828        10     
reco:pandoraWriter:StandardPandora                 0.369669      0.520503      0.605122      0.526297      0.0712935       10     
reco:pandoraTrack:LArPandoraTrackCreation           4.98804       8.99671       12.0546       9.20477       1.93915        10     
reco:pandoraShower:LArPandoraShowerCreation          5.178        11.042        15.8213       10.9746       3.04007        10     
reco:pandoracalo:Calorimetry                        2.06684       4.20699       5.90125       4.24629       1.07653        10     
reco:pandoracalonosce:Calorimetry                   1.88834       3.90232       5.51823       3.92354       1.02507        10     
reco:pandorapid:Chi2ParticleID                    0.00506802    0.00636429    0.00744853     0.0063237    0.000697896      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0524713     0.0680007     0.0819394     0.0699926    0.00858476       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00465615    0.00595473    0.00700799    0.00589707    0.000627241      10     
reco:pandoraShowercalo:ShowerCalorimetry            4.31369       9.88857       15.0867       10.1912       3.34227        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.15188       9.67321       14.5528       9.82528       3.3631         10     
reco:emtrkmichelid:EmTrackMichelId                  153.479       213.514       275.881       212.772       34.2877        10     
reco:crttag:SingleCRTMatchingProducer              0.0218152     0.0301235     0.0382063     0.0301105    0.00518072       10     
reco:crtreco:TwoCRTMatchingProducer                0.0186601     0.024993      0.0303975     0.0247988    0.00357918       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.10795       1.31357       1.52152       1.34786      0.131604        10     
reco:pandora2Track:LArPandoraTrackCreation          16.4234       31.0712       40.4874       33.0351       7.52975        10     
reco:pandora2calo:Calorimetry                       6.03242       13.4645       19.4883       13.8872       4.02108        10     
reco:pandora2calonosce:Calorimetry                  5.61972       12.3805       17.3211       12.2359       3.54147        10     
reco:pandora2pid:Chi2ParticleID                    0.0110431     0.0160769     0.0228192     0.0158576    0.00317533       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0642032     0.0889907     0.117143      0.0886696     0.0154092       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0107968     0.0157157     0.022661      0.0154163    0.00317304       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.68019       10.0503       13.0788       10.7181       2.21142        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.37069       11.1658       15.1973       11.483        3.05072        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.6605        9.89869       13.3066       10.4726       2.58778        10     
[art]:TriggerResults:TriggerResultInserter        5.6739e-05    7.06695e-05   0.000129656   6.31055e-05   2.06004e-05      10     
end_path:out1:RootOutput                           8.596e-06    1.01639e-05   1.4411e-05      9.5e-06     1.7231e-06       10     
end_path:out1:RootOutput(write)                     2.20429       3.14704       3.97388       3.06344      0.476904        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4760.09 MB
  Peak resident set size usage (VmHWM): 3959.14 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.30 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.75 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.28 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9156792
-rw-r--r-- 1 pildune32 pildune 2182400748 Nov 20 11:51 detsim_stage2.root
-rw-r--r-- 1 pildune32 pildune 2075136672 Nov 20 11:49 detsim_stage1.root
-rw-r--r-- 1 pildune32 pildune 1837546833 Nov 20 13:13 reco_stage1.root
-rw-r--r-- 1 pildune32 pildune 1632175845 Nov 20 10:21 g4_stage2.root
-rw-r--r-- 1 pildune32 pildune 1580706510 Nov 20 10:07 g4_stage1.root
-rw-r--r-- 1 pildune32 pildune   67974740 Nov 20 13:07 Pandora_Events.pndr
-rw-r--r-- 1 pildune32 pildune     242972 Nov 20 13:13 jobscript.log
-rw-r--r-- 1 pildune32 pildune     217088 Nov 20 13:13 mem.db
-rw-r--r-- 1 pildune32 pildune      62376 Nov 20 13:13 Pandora_Geometry.xml
-rw-r--r-- 1 pildune32 pildune      36864 Nov 20 13:13 time.db
-rw-r--r-- 1 pildune32 pildune        519 Nov 20 11:51 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Nov 20 10:21 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Nov 20 13:13 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune32 pildune         52 Nov 20 10:01 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune          7 Nov 20 13:13 justin-processed-pfns.txt
-rw-r--r-- 1 pildune32 pildune          0 Nov 20 11:58 ProtoDUNE.log
-rw-r--r-- 1 pildune32 pildune          0 Nov 20 11:51 debugprod.log
justIN time: 2026-02-09 03:29:19 UTC       justIN version: 01.06.00