justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 294675.1@dunegpschedd02.fnal.gov

Jobsub ID294675.1@dunegpschedd02.fnal.gov
Workflow ID12383
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-24 21:42:03
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce503
Last heartbeat2026-01-25 01:35:54
From worker nodeHostnameb9p05p0564.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-24 21:42:38
Input filesmonte-carlo-012383-000001
JobscriptExit code0
Real time3h (13904s)
CPU time3h (13270s = 95%)
Max RSS bytes4361179136 (4159 MiB)
Outputting started2026-01-25 01:34:23
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12383/1/001/reco_stage1.root
Finished2026-01-25 01:35:54
Saved logsjustin-logs:294675.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 129709 traces, 0 summary
Retagger: tagging trace set: gauss with 101225 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 101225 traces tagged "gauss"
FrameSaver: q=3.60527e+06 n=1825715 tag=gauss
wclsFrameSaver: saving 129709 traces tagged "wiener"
FrameSaver: q=3.8276e+06 n=1635955 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 28336 XUs and 35351 XVs -> 433 XUVs
C:0 T:1 56117 XUs and 69709 XVs -> 5760 XUVs
C:0 T:2 64551 XUs and 103163 XVs -> 3373 XUVs
C:0 T:3 29167 XUs and 42180 XVs -> 242 XUVs
C:0 T:4 8306 XUs and 11417 XVs -> 84 XUVs
C:0 T:5 29859 XUs and 37400 XVs -> 4647 XUVs
C:0 T:6 134141 XUs and 204350 XVs -> 6009 XUVs
C:0 T:7 51438 XUs and 75844 XVs -> 739 XUVs
C:0 T:8 14436 XUs and 14459 XVs -> 282 XUVs
C:0 T:9 141217 XUs and 187093 XVs -> 26445 XUVs
C:0 T:10 8643 XUs and 24483 XVs -> 1872 XUVs
C:0 T:11 33 XUs and 15 XVs -> 7 XUVs
49893 XUVs total
12138 collection wire objects
46266 potential space points
Neighbour search...
8420752 tests to find 3549592 neighbours
Iterating with no regularization...
Begin: 9.22296e+07
0 7.23001e+07
1 7.06108e+07
2 7.03861e+07
3 7.03184e+07
Now with regularization...
Begin: 5.81903e+07
0 5.80936e+07
1 5.8053e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411945070
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411945070
fLifetime: 30000 [us]
25-Jan-2026 02:34:21 CET  Closed output file "reco_stage1.root"
25-Jan-2026 02:34:21 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          246.03        368.046       486.105       354.652       77.7598        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                             0.0013544     0.0036692    0.00651461     0.0035043    0.00155372       10     
reco:ophit:OpHitFinder                             0.226662      0.340235      0.461567      0.341166      0.0660525       10     
reco:opflash:OpFlashFinder                         0.132997      0.200581      0.289409       0.20294      0.0456274       10     
reco:opslicer:OpSlicer                             0.387331      0.734359       1.06106      0.719062      0.196957        10     
reco:caldata:DataPrepModule                         4.26724       5.98851       7.28503       5.9886        1.0603         10     
reco:wclsdatasp:WireCellToolkit                     29.9867       35.4765       40.0925       35.5758       2.85169        10     
reco:gaushit:GausHitFinder                         0.883348       1.35077       1.69152       1.44764      0.282313        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000363504   0.000600918   0.000942211   0.000569317   0.000157455      10     
reco:reco3d:SpacePointSolver                        9.64414       18.4536       28.2128       19.1289       6.21745        10     
reco:hitpdune:DisambigFromSpacePoints              0.971188       1.88315       2.96096       1.91795      0.609167        10     
reco:pandora:StandardPandora                        20.5309       39.5614       76.2988       35.7113       19.0421        10     
reco:pandoraWriter:StandardPandora                 0.230427      0.433417       0.60227      0.439626      0.102891        10     
reco:pandoraTrack:LArPandoraTrackCreation           1.53772       3.67715       5.50852       3.63233       1.00697        10     
reco:pandoraShower:LArPandoraShowerCreation         1.55816       4.20988       7.10469       3.86745       1.71482        10     
reco:pandoracalo:Calorimetry                       0.868041       2.15605       3.02493       2.20818       0.56667        10     
reco:pandoracalonosce:Calorimetry                  0.706369       1.82668       2.31254       2.01448      0.482284        10     
reco:pandorapid:Chi2ParticleID                    0.00277145    0.00532235    0.00940917     0.0049338    0.00172757       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0275441     0.0483225     0.066007       0.04784      0.0123784       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00259741    0.00492992    0.00718755    0.00455615    0.00148181       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.6319        4.27638       6.99105       4.56866       1.71057        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.7228        4.3348        7.35243       4.21317       1.7221         10     
reco:emtrkmichelid:EmTrackMichelId                  138.549       199.191       247.955       189.811       35.5937        10     
reco:crttag:SingleCRTMatchingProducer             0.00929115     0.0154434     0.0213744     0.0151793    0.00369131       10     
reco:crtreco:TwoCRTMatchingProducer               0.00566937    0.00853923     0.0109567    0.00854195    0.00178334       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.624463       0.94984       1.32064      0.928224       0.20469        10     
reco:pandora2Track:LArPandoraTrackCreation          6.55928       11.8028       16.2867       11.715        3.13946        10     
reco:pandora2calo:Calorimetry                       3.2724        6.03819       8.16525       6.15831       1.75487        10     
reco:pandora2calonosce:Calorimetry                  2.99179       6.2336        9.07805       6.38924       2.01363        10     
reco:pandora2pid:Chi2ParticleID                   0.00922992     0.0168726     0.0232697     0.0180419    0.00459083       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0450596     0.0805416     0.108989      0.0833358     0.0214962       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.009621      0.0164962     0.0218009     0.0179408     0.0045657       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.47233       4.37567       6.78193       4.14359       1.4513         10     
reco:pandora2Showercalo:ShowerCalorimetry           2.7718        5.60059       8.66536       5.43996       1.68305        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.58211       5.50027       8.11754       5.29427       1.84026        10     
[art]:TriggerResults:TriggerResultInserter        5.1571e-05    7.65411e-05   0.000204903   6.29905e-05   4.32144e-05      10     
end_path:out1:RootOutput                           8.27e-06     1.06672e-05    1.891e-05     9.795e-06    2.91254e-06      10     
end_path:out1:RootOutput(write)                     2.15024       3.16833       4.20684       3.07935      0.764989        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5197.41 MB
  Peak resident set size usage (VmHWM): 4361.18 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.27 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.77 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10712864
-rw-r--r--. 1 duneprd np-comp 2496496818 Jan 25 01:30 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2372886282 Jan 25 01:28 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2192360212 Jan 25 02:34 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1957917855 Jan 24 23:06 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1872176121 Jan 24 22:49 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   77517680 Jan 25 02:30 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242914 Jan 25 02:34 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 25 02:34 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 25 02:34 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 25 02:34 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 25 01:30 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 24 23:06 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 25 02:34 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 24 22:42 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 25 02:34 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 25 01:37 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 25 01:30 debugprod.log
justIN time: 2026-02-05 01:20:15 UTC       justIN version: 01.06.00