justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 264455.1@dunegpschedd02.fnal.gov

Jobsub ID264455.1@dunegpschedd02.fnal.gov
Workflow ID11350
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-26 18:47:47
SiteUK_Brunel
EntryCMSHTPC_T2_UK_London_Brunel_dc2_22
Last heartbeat2025-12-26 23:01:15
From worker nodeHostnamewn-a2-21.brunel.ac.uk
cpuinfoIntel(R) Xeon(R) CPU E5-2683 v4 @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-26 18:48:18
Input filesmonte-carlo-011350-000001
JobscriptExit code0
Real time4h (15083s)
CPU time4h (14653s = 97%)
Max RSS bytes4592545792 (4379 MiB)
Outputting started2025-12-26 22:59:42
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11350/1/001/reco_stage1.root
Finished2025-12-26 23:01:15
Saved logsjustin-logs:264455.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

llSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 167040 traces, 0 summary
Retagger: tagging trace set: gauss with 125050 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 125050 traces tagged "gauss"
FrameSaver: q=4.48585e+06 n=2181503 tag=gauss
wclsFrameSaver: saving 167040 traces tagged "wiener"
FrameSaver: q=4.77908e+06 n=1931272 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 59236 XUs and 52891 XVs -> 845 XUVs
C:0 T:1 128452 XUs and 119034 XVs -> 10315 XUVs
C:0 T:2 141599 XUs and 181203 XVs -> 8443 XUVs
C:0 T:3 65095 XUs and 64871 XVs -> 688 XUVs
C:0 T:4 15169 XUs and 13488 XVs -> 188 XUVs
C:0 T:5 56835 XUs and 40634 XVs -> 5253 XUVs
C:0 T:6 119716 XUs and 136745 XVs -> 5980 XUVs
C:0 T:7 46550 XUs and 53115 XVs -> 425 XUVs
C:0 T:8 10957 XUs and 10622 XVs -> 227 XUVs
C:0 T:9 113121 XUs and 91278 XVs -> 11039 XUVs
C:0 T:10 61935 XUs and 19259 XVs -> 4711 XUVs
C:0 T:11 187 XUs and 120 XVs -> 15 XUVs
48129 XUVs total
14395 collection wire objects
43515 potential space points
Neighbour search...
3461947 tests to find 1690326 neighbours
Iterating with no regularization...
Begin: 1.0498e+08
0 8.08768e+07
1 7.93739e+07
2 7.91671e+07
3 7.91022e+07
Now with regularization...
Begin: 6.58872e+07
0 6.58802e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411359218
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411359218
fLifetime: 30000 [us]
26-Dec-2025 22:59:28 GMT  Closed output file "reco_stage1.root"
26-Dec-2025 22:59:28 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          369.285       571.57        700.797       591.756       101.785        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000907509   0.00360596    0.00574285    0.00392192    0.00135224       10     
reco:ophit:OpHitFinder                             0.297378      0.363685      0.573965      0.324703      0.0808144       10     
reco:opflash:OpFlashFinder                         0.198923      0.267936      0.356605      0.254173      0.0500008       10     
reco:opslicer:OpSlicer                              0.68231      0.872673       1.21687      0.787613      0.172654        10     
reco:caldata:DataPrepModule                         6.0444        6.39817       6.8922        6.36748       0.20264        10     
reco:wclsdatasp:WireCellToolkit                     48.2893       53.0764       57.6323       52.6422       2.48409        10     
reco:gaushit:GausHitFinder                          1.27248       1.76884       2.3599        1.77137      0.291626        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00058077    0.000842765   0.00114034    0.000835569   0.000170395      10     
reco:reco3d:SpacePointSolver                        13.7793       24.6135       40.1967       25.1675       7.29106        10     
reco:hitpdune:DisambigFromSpacePoints               1.9328        3.71563       5.23901       3.63356       0.93119        10     
reco:pandora:StandardPandora                        37.8348       66.5045       102.181       63.1173       18.3383        10     
reco:pandoraWriter:StandardPandora                 0.395737      0.620164      0.896916      0.612248      0.126744        10     
reco:pandoraTrack:LArPandoraTrackCreation           5.52852       10.3226       16.121        10.1661       3.03374        10     
reco:pandoraShower:LArPandoraShowerCreation         3.21378       11.0714       19.3755       11.0832       4.20173        10     
reco:pandoracalo:Calorimetry                        2.25413       4.6771        6.78947       4.80358       1.25373        10     
reco:pandoracalonosce:Calorimetry                   2.26023       4.45322       6.57543       4.46117       1.29343        10     
reco:pandorapid:Chi2ParticleID                    0.00640553    0.00805432    0.00914436    0.00861367    0.00106543       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0535645     0.0884988     0.104114      0.0910593     0.0140971       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00615383    0.00781592    0.00937945    0.00795895    0.00108591       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.93773       10.8459       17.7455       10.8981       3.9622         10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.99486       10.2787       13.7768       10.9171       3.22933        10     
reco:emtrkmichelid:EmTrackMichelId                  187.965       260.727       315.24        255.669       41.0367        10     
reco:crttag:SingleCRTMatchingProducer              0.0139369     0.0273359     0.0397684     0.0272337    0.00718198       10     
reco:crtreco:TwoCRTMatchingProducer               0.00882283     0.0210267     0.0333236     0.020794     0.00750508       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.31421       2.3133        3.05417       2.39803       0.51706        10     
reco:pandora2Track:LArPandoraTrackCreation          11.1833       33.3064       46.2395       34.6528       10.9836        10     
reco:pandora2calo:Calorimetry                       5.04107       14.6978       18.6818       16.0949       4.09793        10     
reco:pandora2calonosce:Calorimetry                  4.49138       13.547        17.7141       14.5284       3.95464        10     
reco:pandora2pid:Chi2ParticleID                    0.0139538     0.0242246     0.0368407     0.0237136    0.00615502       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0704882     0.127522       0.17795      0.125896      0.0301564       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0137762     0.0238668     0.0374259     0.0232358    0.00627333       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.7337        9.96618       12.1117       11.0731       2.50123        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.30606       11.9767       18.547        12.2557       3.65557        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.02679       11.0256       14.3594       12.1794       3.07283        10     
[art]:TriggerResults:TriggerResultInserter        4.0899e-05    6.17189e-05   0.000138773   5.1785e-05    2.69938e-05      10     
end_path:out1:RootOutput                           7.922e-06    1.05793e-05    1.784e-05    9.3935e-06    2.98236e-06      10     
end_path:out1:RootOutput(write)                     2.31127       3.68338       4.5249        3.7462       0.620742        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5404.91 MB
  Peak resident set size usage (VmHWM): 4592.55 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.24 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.03 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.18 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11687532
-rw-r--r--. 1 dune000 dune 2695991829 Dec 26 21:21 detsim_stage2.root
-rw-r--r--. 1 dune000 dune 2559371077 Dec 26 21:17 detsim_stage1.root
-rw-r--r--. 1 dune000 dune 2432761657 Dec 26 22:59 reco_stage1.root
-rw-r--r--. 1 dune000 dune 2141853492 Dec 26 19:15 g4_stage2.root
-rw-r--r--. 1 dune000 dune 2050424426 Dec 26 18:56 g4_stage1.root
-rw-r--r--. 1 dune000 dune   87000560 Dec 26 22:51 Pandora_Events.pndr
-rw-r--r--. 1 dune000 dune     242706 Dec 26 22:59 jobscript.log
-rw-r--r--. 1 dune000 dune     217088 Dec 26 22:59 mem.db
-rw-r--r--. 1 dune000 dune      62376 Dec 26 22:59 Pandora_Geometry.xml
-rw-r--r--. 1 dune000 dune      36864 Dec 26 22:59 time.db
-rw-r--r--. 1 dune000 dune        519 Dec 26 21:21 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune000 dune        519 Dec 26 19:15 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune000 dune        519 Dec 26 22:59 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune000 dune         52 Dec 26 18:48 all-input-dids.txt
-rw-r--r--. 1 dune000 dune          7 Dec 26 22:59 justin-processed-pfns.txt
-rw-r--r--. 1 dune000 dune          0 Dec 26 21:30 ProtoDUNE.log
-rw-r--r--. 1 dune000 dune          0 Dec 26 21:21 debugprod.log
justIN time: 2026-02-07 03:33:35 UTC       justIN version: 01.06.00