justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 294673.0@dunegpschedd02.fnal.gov

Jobsub ID294673.0@dunegpschedd02.fnal.gov
Workflow ID12381
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-24 21:42:02
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce503
Last heartbeat2026-01-25 02:07:18
From worker nodeHostnameb9p05p0564.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-24 21:42:59
Input filesmonte-carlo-012381-000001
JobscriptExit code0
Real time4h (15759s)
CPU time4h (15162s = 96%)
Max RSS bytes4623572992 (4409 MiB)
Outputting started2026-01-25 02:05:39
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12381/1/001/reco_stage1.root
Finished2026-01-25 02:07:18
Saved logsjustin-logs:294673.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 216130 traces, 0 summary
Retagger: tagging trace set: gauss with 161718 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 161718 traces tagged "gauss"
FrameSaver: q=5.78923e+06 n=2865348 tag=gauss
wclsFrameSaver: saving 216130 traces tagged "wiener"
FrameSaver: q=6.12866e+06 n=2527919 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 51688 XUs and 59005 XVs -> 1053 XUVs
C:0 T:1 118946 XUs and 118387 XVs -> 7303 XUVs
C:0 T:2 196932 XUs and 230691 XVs -> 8961 XUVs
C:0 T:3 90055 XUs and 84059 XVs -> 1113 XUVs
C:0 T:4 16142 XUs and 21457 XVs -> 193 XUVs
C:0 T:5 63312 XUs and 64848 XVs -> 5447 XUVs
C:0 T:6 139775 XUs and 188341 XVs -> 13758 XUVs
C:0 T:7 48488 XUs and 62648 XVs -> 894 XUVs
C:0 T:8 9309 XUs and 9625 XVs -> 170 XUVs
C:0 T:9 78885 XUs and 108063 XVs -> 15559 XUVs
C:0 T:10 95100 XUs and 102037 XVs -> 10811 XUVs
C:0 T:11 353 XUs and 279 XVs -> 32 XUVs
65294 XUVs total
19789 collection wire objects
59374 potential space points
Neighbour search...
6407638 tests to find 3018444 neighbours
Iterating with no regularization...
Begin: 1.1326e+08
0 9.41425e+07
1 9.26983e+07
2 9.25056e+07
3 9.24551e+07
Now with regularization...
Begin: 7.51286e+07
0 7.50715e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411945104
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411945104
fLifetime: 30000 [us]
25-Jan-2026 03:05:37 CET  Closed output file "reco_stage1.root"
25-Jan-2026 03:05:37 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          265.334       407.709       612.494       400.025       109.489        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000764279   0.00348546    0.00562393    0.00391784    0.00145673       10     
reco:ophit:OpHitFinder                              0.24482      0.314953       0.44249      0.284187       0.06811        10     
reco:opflash:OpFlashFinder                         0.168046      0.222879      0.286136      0.211628      0.0389247       10     
reco:opslicer:OpSlicer                             0.469144      0.728554       1.08374      0.682259      0.196995        10     
reco:caldata:DataPrepModule                         3.94477       5.29678       6.94113       5.07849      0.981162        10     
reco:wclsdatasp:WireCellToolkit                     29.5642       34.0481        38.41        34.4443       2.82495        10     
reco:gaushit:GausHitFinder                         0.866397       1.48448       2.03503       1.57424      0.416654        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000335254   0.000568491   0.000750799   0.000558942   0.000129623      10     
reco:reco3d:SpacePointSolver                        7.33823       20.2689       32.7957       21.0196       7.80215        10     
reco:hitpdune:DisambigFromSpacePoints               1.03442       2.22356       3.85815       2.02883      0.868954        10     
reco:pandora:StandardPandora                        19.958        47.4323       89.7556       46.0811       23.7835        10     
reco:pandoraWriter:StandardPandora                 0.261623      0.491832      0.877375      0.456716      0.195731        10     
reco:pandoraTrack:LArPandoraTrackCreation           2.18882       4.50292       7.76359       3.93909       1.88748        10     
reco:pandoraShower:LArPandoraShowerCreation         1.92801       5.50011       10.9134       5.12363       2.65055        10     
reco:pandoracalo:Calorimetry                        1.42826       2.54001       3.95609       2.23975      0.839375        10     
reco:pandoracalonosce:Calorimetry                   1.31285       2.31207       3.56941       2.03578      0.694281        10     
reco:pandorapid:Chi2ParticleID                    0.00388689    0.00635659     0.0117855    0.00574355    0.00235112       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0337266     0.0603249     0.0922522     0.0583744     0.0186835       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00368688    0.00603878     0.0100913    0.00567813    0.00203629       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.18779       5.51271       10.298        5.15516       2.4433         10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.19229       5.47195       11.9562       5.06803       2.72183        10     
reco:emtrkmichelid:EmTrackMichelId                  144.493       214.805       309.592       213.308       48.0902        10     
reco:crttag:SingleCRTMatchingProducer             0.00817077     0.0149253     0.0231044     0.0135598    0.00453949       10     
reco:crtreco:TwoCRTMatchingProducer               0.00547214    0.00830796     0.0126427    0.00846587    0.00190006       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.549891       1.03536       1.49332       1.04741      0.251611        10     
reco:pandora2Track:LArPandoraTrackCreation          7.51755       16.1238       29.3438       13.8219       6.8701         10     
reco:pandora2calo:Calorimetry                       5.02778       7.93001       13.2977       7.36466       2.38867        10     
reco:pandora2calonosce:Calorimetry                  3.60685       7.54296       13.212        6.89712       2.7148         10     
reco:pandora2pid:Chi2ParticleID                   0.00816865     0.0165967     0.0283303     0.0151983    0.00675083       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0419097     0.0806508     0.138003      0.0727573     0.030702        10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00809152     0.0160603     0.0269203     0.014871     0.00638545       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.24424       5.39515       11.0763       4.81268       2.42618        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.96682       6.72973       9.64828       6.7291        2.20212        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.74913       6.3137        9.86542       6.27278       2.1993         10     
[art]:TriggerResults:TriggerResultInserter         4.266e-05    6.49889e-05   0.000155742   5.4726e-05    3.19292e-05      10     
end_path:out1:RootOutput                           7.18e-06     1.30652e-05    3.455e-05    1.0581e-05    7.40849e-06      10     
end_path:out1:RootOutput(write)                     2.24896       3.17757       4.57789       3.0177        0.71381        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5458.23 MB
  Peak resident set size usage (VmHWM): 4623.57 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.45 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11818372
-rw-r--r--. 1 duneprd np-comp 2712151049 Jan 25 01:55 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2570937623 Jan 25 01:53 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2458596885 Jan 25 03:05 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 2176586627 Jan 24 23:08 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 2093341399 Jan 24 22:50 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   89652770 Jan 25 02:58 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242704 Jan 25 03:05 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 25 03:05 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 25 03:05 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 25 03:05 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 25 01:55 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 24 23:08 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 25 03:05 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 24 22:43 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 25 03:05 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 25 02:04 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 25 01:55 debugprod.log
justIN time: 2026-02-05 01:19:47 UTC       justIN version: 01.06.00