justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 247869.0@dunegpschedd01.fnal.gov

Jobsub ID247869.0@dunegpschedd01.fnal.gov
Workflow ID9991
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-12 12:22:26
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-11-12 18:41:38
From worker nodeHostnamewn1906251.tier2.hep.manchester.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-12 12:23:35
Input filesmonte-carlo-009991-000001
JobscriptExit code0
Real time5h (21536s)
CPU time5h (20945s = 97%)
Max RSS bytes4089847808 (3900 MiB)
Outputting started2025-11-12 18:22:32
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09991/1/001/reco_stage1.root
Finished2025-11-12 18:41:38
Saved logsjustin-logs:247869.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 125535 traces, 0 summary
Retagger: tagging trace set: gauss with 96678 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 96678 traces tagged "gauss"
FrameSaver: q=3.59008e+06 n=1725271 tag=gauss
wclsFrameSaver: saving 125535 traces tagged "wiener"
FrameSaver: q=3.80006e+06 n=1535089 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 15295 XUs and 15615 XVs -> 93 XUVs
C:0 T:1 21361 XUs and 21254 XVs -> 1245 XUVs
C:0 T:2 136461 XUs and 99767 XVs -> 16409 XUVs
C:0 T:3 47986 XUs and 34640 XVs -> 1108 XUVs
C:0 T:4 11559 XUs and 10847 XVs -> 86 XUVs
C:0 T:5 38162 XUs and 26057 XVs -> 2271 XUVs
C:0 T:6 163460 XUs and 160853 XVs -> 8619 XUVs
C:0 T:7 54669 XUs and 57670 XVs -> 959 XUVs
C:0 T:8 9174 XUs and 6851 XVs -> 115 XUVs
C:0 T:9 58970 XUs and 48814 XVs -> 3309 XUVs
C:0 T:10 49568 XUs and 31569 XVs -> 5747 XUVs
C:0 T:11 563 XUs and 425 XVs -> 33 XUVs
39994 XUVs total
9849 collection wire objects
34747 potential space points
Neighbour search...
4347527 tests to find 1987410 neighbours
Iterating with no regularization...
Begin: 7.97171e+07
0 6.72443e+07
1 6.6281e+07
2 6.61325e+07
3 6.60872e+07
Now with regularization...
Begin: 5.57614e+07
0 5.57104e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410468609
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410468609
fLifetime: 30000 [us]
12-Nov-2025 18:22:28 GMT  Closed output file "reco_stage1.root"
12-Nov-2025 18:22:28 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          574.404       777.128       1022.18        805.2        142.595        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00197254    0.00663831     0.020173     0.00479441    0.00510408       10     
reco:ophit:OpHitFinder                             0.395173      0.495047      0.544287      0.511184      0.0476607       10     
reco:opflash:OpFlashFinder                         0.194534      0.274288      0.353773      0.275325      0.0406398       10     
reco:opslicer:OpSlicer                             0.878276       1.16424       1.32724       1.23522      0.160754        10     
reco:caldata:DataPrepModule                         10.7792       11.2892       11.6674       11.3742      0.322905        10     
reco:wclsdatasp:WireCellToolkit                     73.7307       80.2426       82.5178       81.1187       2.53984        10     
reco:gaushit:GausHitFinder                          1.62282       2.49947       3.25202       2.58681      0.501883        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000735746   0.000937425    0.0011654    0.000968902    0.0001336       10     
reco:reco3d:SpacePointSolver                        15.4619       27.1157       37.6541       27.4574       5.80574        10     
reco:hitpdune:DisambigFromSpacePoints               2.4737        4.12637       5.80069       4.36988       1.12966        10     
reco:pandora:StandardPandora                        36.1577       65.7521       104.058       67.622        21.7371        10     
reco:pandoraWriter:StandardPandora                 0.715253      0.873885       1.06037       0.89694      0.112503        10     
reco:pandoraTrack:LArPandoraTrackCreation           9.18134       13.3868       17.9377       13.4448       2.86347        10     
reco:pandoraShower:LArPandoraShowerCreation         7.91706       14.1732       22.1258       14.4049       4.67235        10     
reco:pandoracalo:Calorimetry                        3.96451       6.36834       8.5458        6.78569       1.44945        10     
reco:pandoracalonosce:Calorimetry                   3.64244       6.10825       8.61094       6.40543       1.61773        10     
reco:pandorapid:Chi2ParticleID                    0.00936542      0.01184      0.0134145     0.0118759    0.00115276       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP          0.10337      0.119316      0.134884      0.120923      0.0101219       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00896582     0.0109987     0.0132063     0.0111532    0.00128253       10     
reco:pandoraShowercalo:ShowerCalorimetry            7.20476       14.1635       23.519        15.1205       5.44144        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       6.87322       13.6861       22.5689       14.3525       5.13205        10     
reco:emtrkmichelid:EmTrackMichelId                  299.693       384.349       469.742       389.05        55.7548        10     
reco:crttag:SingleCRTMatchingProducer              0.0270733     0.0504889     0.0649817     0.0533624     0.0111446       10     
reco:crtreco:TwoCRTMatchingProducer                0.0183445     0.0309284     0.0436746     0.0307511    0.00720486       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.57288       1.90399       2.39129       1.91624       0.26399        10     
reco:pandora2Track:LArPandoraTrackCreation          24.4799       43.3086       67.5295       45.7891       13.5082        10     
reco:pandora2calo:Calorimetry                       9.75778       19.0821       30.5849       20.7716       6.52258        10     
reco:pandora2calonosce:Calorimetry                  9.50685       17.7922       28.5626       19.3112       6.06454        10     
reco:pandora2pid:Chi2ParticleID                    0.0164848     0.0295548     0.0455445     0.0295038    0.00795863       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0939089     0.159675      0.235676      0.153184      0.0370573       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0132368     0.0259062      0.03515      0.0284978    0.00699445       10     
reco:pandora2Shower:LArPandoraShowerCreation        8.03047       13.2835       19.646        14.1664       3.76689        10     
reco:pandora2Showercalo:ShowerCalorimetry           8.42463       15.9956       24.6081       17.301        5.16877        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      7.71696       13.9723       21.1875       15.5757       4.42808        10     
[art]:TriggerResults:TriggerResultInserter        0.000111846   0.000142964   0.000284218   0.000126875   4.92266e-05      10     
end_path:out1:RootOutput                          1.2227e-05    1.57231e-05   2.8769e-05    1.4438e-05    4.6521e-06       10     
end_path:out1:RootOutput(write)                     4.13265       5.06692       6.06645       5.19675      0.700279        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4921.28 MB
  Peak resident set size usage (VmHWM): 4089.85 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.44 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   6.96 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.40 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9780740
-rw-r--r-- 1 dune01 dune01 2321774412 Nov 12 16:09 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2207781952 Nov 12 16:05 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 1996723113 Nov 12 18:22 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1749898945 Nov 12 12:53 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1666390792 Nov 12 12:33 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   72277940 Nov 12 18:12 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242919 Nov 12 18:22 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Nov 12 18:22 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Nov 12 18:22 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Nov 12 18:22 time.db
-rw-r--r-- 1 dune01 dune01        519 Nov 12 16:09 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 12 12:53 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 12 18:22 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Nov 12 12:23 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Nov 12 18:22 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Nov 12 16:18 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Nov 12 16:09 debugprod.log
justIN time: 2026-02-10 01:17:48 UTC       justIN version: 01.06.00