justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 271427.1@dunegpschedd01.fnal.gov

Jobsub ID271427.1@dunegpschedd01.fnal.gov
Workflow ID11486
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-03 13:49:50
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce512
Last heartbeat2026-01-03 17:31:15
From worker nodeHostnameb9p09p1103.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-03 13:51:18
Input filesmonte-carlo-011486-000001
JobscriptExit code0
Real time3h (13113s)
CPU time3h (11981s = 91%)
Max RSS bytes4210925568 (4015 MiB)
Outputting started2026-01-03 17:29:52
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11486/1/001/reco_stage1.root
Finished2026-01-03 17:31:15
Saved logsjustin-logs:271427.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ng as is
Retagger: tagging trace set: wiener with 151145 traces, 0 summary
Retagger: tagging trace set: gauss with 115373 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 115373 traces tagged "gauss"
FrameSaver: q=4.1539e+06 n=2102923 tag=gauss
wclsFrameSaver: saving 151145 traces tagged "wiener"
FrameSaver: q=4.42582e+06 n=1872437 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 23444 XUs and 26497 XVs -> 308 XUVs
C:0 T:1 35236 XUs and 37989 XVs -> 2641 XUVs
C:0 T:2 174711 XUs and 279934 XVs -> 25960 XUVs
C:0 T:3 58642 XUs and 78367 XVs -> 1522 XUVs
C:0 T:4 18081 XUs and 19438 XVs -> 134 XUVs
C:0 T:5 80939 XUs and 87936 XVs -> 6772 XUVs
C:0 T:6 161710 XUs and 194411 XVs -> 27126 XUVs
C:0 T:7 48787 XUs and 48321 XVs -> 770 XUVs
C:0 T:8 9894 XUs and 10015 XVs -> 228 XUVs
C:0 T:9 79348 XUs and 167807 XVs -> 8362 XUVs
C:0 T:10 47024 XUs and 46374 XVs -> 10585 XUVs
C:0 T:11 90 XUs and 51 XVs -> 5 XUVs
84413 XUVs total
15300 collection wire objects
77504 potential space points
Neighbour search...
20066322 tests to find 7769832 neighbours
Iterating with no regularization...
Begin: 1.2114e+08
0 8.9687e+07
1 8.75878e+07
2 8.72987e+07
3 8.72235e+07
Now with regularization...
Begin: 7.24451e+07
0 7.23388e+07
1 7.22887e+07
TestBeamParticleCreationAlgorithm::SetupTestBeamVertex - Test beam particle has no initial vertex
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411515922
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411515922
fLifetime: 30000 [us]
03-Jan-2026 18:29:50 CET  Closed output file "reco_stage1.root"
03-Jan-2026 18:29:50 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          241.93        345.404       452.243       348.46        67.1431        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000763255   0.00339241     0.0059294    0.00337045    0.00152547       10     
reco:ophit:OpHitFinder                             0.223222      0.268915      0.370774       0.25294      0.0449848       10     
reco:opflash:OpFlashFinder                         0.139559      0.171086      0.205085       0.16503      0.0238952       10     
reco:opslicer:OpSlicer                             0.487043      0.626291      0.892978      0.590952      0.113845        10     
reco:caldata:DataPrepModule                         4.02807       5.24795       7.05276       4.81256       1.01883        10     
reco:wclsdatasp:WireCellToolkit                     28.1561       30.8601       35.3465       30.389        2.38393        10     
reco:gaushit:GausHitFinder                         0.680343       1.08276       1.40738       1.13679      0.225339        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000334162   0.000459639   0.000625734   0.000448963   7.81607e-05      10     
reco:reco3d:SpacePointSolver                        8.58293       18.1293       30.389        15.3599       6.38842        10     
reco:hitpdune:DisambigFromSpacePoints              0.853035       1.61813       2.84051       1.55829      0.664821        10     
reco:pandora:StandardPandora                        17.5232       40.4118       95.4782       33.4347       20.7801        10     
reco:pandoraWriter:StandardPandora                 0.263405      0.364417      0.468169      0.364563      0.0640372       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.96061       3.27787       4.56978       3.44585      0.762988        10     
reco:pandoraShower:LArPandoraShowerCreation         1.53727       3.53393       4.84719       3.73244       1.07224        10     
reco:pandoracalo:Calorimetry                        1.12464       2.07072       3.11097       2.14084      0.580687        10     
reco:pandoracalonosce:Calorimetry                   1.03987       1.82016       2.73682       1.7802       0.461365        10     
reco:pandorapid:Chi2ParticleID                     0.0036998    0.00578566    0.00829124    0.00475435    0.00189403       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0320709     0.0524508     0.0783845     0.0475392     0.0167503       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00340039    0.00559397    0.00821702    0.00440522    0.00200351       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.8967        4.28475       7.12712       4.17808       1.40297        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.2317        4.04522       6.12542       4.13121       1.19833        10     
reco:emtrkmichelid:EmTrackMichelId                  138.97        186.373       240.823       191.729       31.7343        10     
reco:crttag:SingleCRTMatchingProducer             0.00979802     0.0145118     0.0215138     0.0153024    0.00345702       10     
reco:crtreco:TwoCRTMatchingProducer               0.00504597    0.00867877     0.0124612    0.00842138    0.00241316       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.667292      0.906853       1.33554      0.862103      0.235228        10     
reco:pandora2Track:LArPandoraTrackCreation          7.14501       11.8486       18.4331       11.4347       3.60914        10     
reco:pandora2calo:Calorimetry                       3.65479       6.17227       9.14319       5.89484       1.76886        10     
reco:pandora2calonosce:Calorimetry                  2.40094       5.68613       8.13704       6.05016       1.45895        10     
reco:pandora2pid:Chi2ParticleID                   0.00776009     0.0138916     0.0208374     0.012429     0.00470951       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0388817     0.0675415     0.104525      0.0669992     0.0218545       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00769279     0.0128509     0.0211591     0.0110081    0.00424171       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.94112       3.91093        5.918        3.80028       1.16874        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.39528       4.95593       7.25121        4.743        1.15225        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.99597       4.66864       6.72198       4.38648       1.20772        10     
[art]:TriggerResults:TriggerResultInserter         4.081e-05    6.13044e-05   0.000132001   5.4155e-05    2.46978e-05      10     
end_path:out1:RootOutput                           8.21e-06     1.0215e-05     1.379e-05     9.155e-06    1.84066e-06      10     
end_path:out1:RootOutput(write)                     1.91393       2.78324       3.73641       2.60226      0.609463        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5040.26 MB
  Peak resident set size usage (VmHWM): 4210.93 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.29 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10419640
-rw-r--r--. 1 duneprd np-comp 2441632848 Jan  3 17:30 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2320476687 Jan  3 17:29 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2145884949 Jan  3 18:29 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1880350574 Jan  3 15:21 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1801065391 Jan  3 15:00 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   79691280 Jan  3 18:25 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     243268 Jan  3 18:29 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan  3 18:29 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan  3 18:29 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan  3 18:29 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan  3 17:30 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan  3 15:21 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan  3 18:29 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan  3 14:51 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan  3 18:29 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan  3 17:34 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan  3 17:30 debugprod.log
justIN time: 2026-02-06 22:18:42 UTC       justIN version: 01.06.00