justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 237155.1@dunegpschedd02.fnal.gov

Jobsub ID237155.1@dunegpschedd02.fnal.gov
Workflow ID9578
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-04 07:51:49
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce02
Last heartbeat2025-11-04 13:09:11
From worker nodeHostnamecn041.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-04 07:53:02
Input filesmonte-carlo-009578-000001
JobscriptExit code0
Real time5h (18757s)
CPU time5h (18457s = 98%)
Max RSS bytes4401037312 (4197 MiB)
Outputting started2025-11-04 13:05:40
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09578/1/001/reco_stage1.root
Finished2025-11-04 13:09:11
Saved logsjustin-logs:237155.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

r: tagging trace set: wiener with 146825 traces, 0 summary
Retagger: tagging trace set: gauss with 107059 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 107059 traces tagged "gauss"
FrameSaver: q=3.71569e+06 n=1853319 tag=gauss
wclsFrameSaver: saving 146825 traces tagged "wiener"
FrameSaver: q=3.97142e+06 n=1617774 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 33262 XUs and 35758 XVs -> 483 XUVs
C:0 T:1 85985 XUs and 74079 XVs -> 5374 XUVs
C:0 T:2 132040 XUs and 108108 XVs -> 13147 XUVs
C:0 T:3 50766 XUs and 38825 XVs -> 494 XUVs
C:0 T:4 11042 XUs and 10628 XVs -> 82 XUVs
C:0 T:5 47231 XUs and 39130 XVs -> 6168 XUVs
C:0 T:6 333043 XUs and 234101 XVs -> 54576 XUVs
C:0 T:7 55192 XUs and 52293 XVs -> 974 XUVs
C:0 T:8 7442 XUs and 6256 XVs -> 390 XUVs
C:0 T:9 106970 XUs and 63321 XVs -> 15218 XUVs
C:0 T:10 111807 XUs and 118531 XVs -> 24158 XUVs
C:0 T:11 497 XUs and 283 XVs -> 50 XUVs
121114 XUVs total
14499 collection wire objects
112394 potential space points
Neighbour search...
33492024 tests to find 13252426 neighbours
Iterating with no regularization...
Begin: 6.98096e+07
0 5.48231e+07
1 5.38632e+07
2 5.36966e+07
3 5.36402e+07
4 5.36121e+07
Now with regularization...
Begin: 4.30006e+07
0 4.29068e+07
1 4.2831e+07
2 4.27696e+07
3 4.27177e+07
4 4.26725e+07
5 4.26318e+07
TestBeamParticleCreationAlgorithm::SetupTestBeamVertex - Test beam particle has no initial vertex
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410304050
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410304050
fLifetime: 30000 [us]
04-Nov-2025 13:05:37 GMT  Closed output file "reco_stage1.root"
04-Nov-2025 13:05:37 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          608.145       881.675       1184.24       822.066       180.525        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00178397     0.0055475     0.0108722    0.00575269    0.00242863       10     
reco:ophit:OpHitFinder                             0.327245      0.419233      0.516484      0.398519      0.0626901       10     
reco:opflash:OpFlashFinder                         0.243911      0.320378      0.440986      0.290741      0.0684927       10     
reco:opslicer:OpSlicer                             0.799363       1.03511        1.374       0.993441      0.182469        10     
reco:caldata:DataPrepModule                         7.14662       8.45425       9.50559       8.3889        0.75899        10     
reco:wclsdatasp:WireCellToolkit                     64.6481       69.8862       76.2731       69.1342       3.98311        10     
reco:gaushit:GausHitFinder                          1.77476       2.3297        3.41459       2.21473      0.412095        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000888799   0.00130035    0.00156329    0.00131659    0.000188882      10     
reco:reco3d:SpacePointSolver                        18.0106       31.5375       69.9151       25.6321       15.7933        10     
reco:hitpdune:DisambigFromSpacePoints               2.91818       4.51354       8.57969       4.14905       1.45021        10     
reco:pandora:StandardPandora                        50.5645       94.5039       208.629       79.1094       45.6737        10     
reco:pandoraWriter:StandardPandora                 0.665023      0.843603      0.995519      0.858576       0.11569        10     
reco:pandoraTrack:LArPandoraTrackCreation           15.915        20.3264       28.5164       19.0702       4.17781        10     
reco:pandoraShower:LArPandoraShowerCreation         11.0255       21.5601       31.8816       20.0724       5.9935         10     
reco:pandoracalo:Calorimetry                        6.20371       10.1568       14.6559       9.9311        2.25408        10     
reco:pandoracalonosce:Calorimetry                   6.19275        9.753        14.5246       9.36818       2.29134        10     
reco:pandorapid:Chi2ParticleID                    0.00874651     0.0115462     0.0161556     0.0114961    0.00224753       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0716601     0.107535      0.164655      0.0951436     0.0300232       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00844032     0.0105489     0.0136948    0.00987357     0.0017442       10     
reco:pandoraShowercalo:ShowerCalorimetry            10.0457       23.4142       35.0405       21.8117       6.91192        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       9.97575       22.9452       34.4366       21.5661       6.85599        10     
reco:emtrkmichelid:EmTrackMichelId                  262.784       345.602       451.456       340.595       60.1576        10     
reco:crttag:SingleCRTMatchingProducer              0.0353565     0.0523271     0.0678458     0.051829      0.0089359       10     
reco:crtreco:TwoCRTMatchingProducer                0.0281874     0.0444407     0.054156      0.0451565    0.00721933       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.43839       1.97192       2.85486       1.92926      0.436024        10     
reco:pandora2Track:LArPandoraTrackCreation          38.6057       66.7945       95.9892       63.9945       16.6552        10     
reco:pandora2calo:Calorimetry                       17.6664       32.9033       48.3935       32.0279       8.83779        10     
reco:pandora2calonosce:Calorimetry                  16.9997       31.6882       45.9584       31.068        8.15359        10     
reco:pandora2pid:Chi2ParticleID                    0.0203567     0.0259054     0.0353053     0.0250553     0.0044219       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.107263      0.147395      0.198495      0.155829      0.0280939       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0202357     0.0255758     0.0344894     0.0245994     0.0045849       10     
reco:pandora2Shower:LArPandoraShowerCreation        13.1398       20.7706       26.4091       20.5339       4.11864        10     
reco:pandora2Showercalo:ShowerCalorimetry           15.6738       28.7352       39.8273       28.6612       6.80778        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      14.285        26.0584       35.8135       25.9711       6.27856        10     
[art]:TriggerResults:TriggerResultInserter        6.8541e-05    0.000105611   0.000205947   8.75035e-05   3.82578e-05      10     
end_path:out1:RootOutput                          1.1704e-05    1.41267e-05    1.902e-05    1.2918e-05     2.735e-06       10     
end_path:out1:RootOutput(write)                     3.16366       4.45972       6.27895       4.2518       0.920961        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5203.31 MB
  Peak resident set size usage (VmHWM): 4401.04 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.34 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.23 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.28 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10734036
-rw-r--r-- 1 pildune22 pildune 2506651851 Nov  4 10:35 detsim_stage2.root
-rw-r--r-- 1 pildune22 pildune 2377009620 Nov  4 10:32 detsim_stage1.root
-rw-r--r-- 1 pildune22 pildune 2220428534 Nov  4 13:05 reco_stage1.root
-rw-r--r-- 1 pildune22 pildune 1943018212 Nov  4 08:24 g4_stage2.root
-rw-r--r-- 1 pildune22 pildune 1862615264 Nov  4 08:01 g4_stage1.root
-rw-r--r-- 1 pildune22 pildune   81314000 Nov  4 12:53 Pandora_Events.pndr
-rw-r--r-- 1 pildune22 pildune     243020 Nov  4 13:05 jobscript.log
-rw-r--r-- 1 pildune22 pildune     217088 Nov  4 13:05 mem.db
-rw-r--r-- 1 pildune22 pildune      62376 Nov  4 13:05 Pandora_Geometry.xml
-rw-r--r-- 1 pildune22 pildune      36864 Nov  4 13:05 time.db
-rw-r--r-- 1 pildune22 pildune        519 Nov  4 10:35 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Nov  4 08:24 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Nov  4 13:05 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune22 pildune         52 Nov  4 07:53 all-input-dids.txt
-rw-r--r-- 1 pildune22 pildune          7 Nov  4 13:05 justin-processed-pfns.txt
-rw-r--r-- 1 pildune22 pildune          0 Nov  4 10:46 ProtoDUNE.log
-rw-r--r-- 1 pildune22 pildune          0 Nov  4 10:35 debugprod.log
justIN time: 2026-02-11 10:48:41 UTC       justIN version: 01.06.00