justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 248947.1@dunegpschedd02.fnal.gov

Jobsub ID248947.1@dunegpschedd02.fnal.gov
Workflow ID10415
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-20 18:58:45
SiteUK_Brunel
EntryCMSHTPC_T2_UK_London_Brunel_dc2_26
Last heartbeat2025-11-20 22:56:26
From worker nodeHostnamewn-a4-03
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-20 19:19:58
Input filesmonte-carlo-010415-000001
JobscriptExit code0
Real time3h (12898s)
CPU time3h (12433s = 96%)
Max RSS bytes4197404672 (4002 MiB)
Outputting started2025-11-20 22:54:57
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10415/1/001/reco_stage1.root
Finished2025-11-20 22:56:26
Saved logsjustin-logs:248947.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

olBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 118352 traces, 0 summary
Retagger: tagging trace set: gauss with 92035 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 92035 traces tagged "gauss"
FrameSaver: q=3.05499e+06 n=1648701 tag=gauss
wclsFrameSaver: saving 118352 traces tagged "wiener"
FrameSaver: q=3.22249e+06 n=1460853 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 37641 XUs and 37163 XVs -> 666 XUVs
C:0 T:1 87733 XUs and 80983 XVs -> 4256 XUVs
C:0 T:2 113871 XUs and 110291 XVs -> 4816 XUVs
C:0 T:3 58783 XUs and 44450 XVs -> 286 XUVs
C:0 T:4 9145 XUs and 6806 XVs -> 57 XUVs
C:0 T:5 28162 XUs and 14905 XVs -> 2019 XUVs
C:0 T:6 184696 XUs and 159220 XVs -> 14824 XUVs
C:0 T:7 48448 XUs and 55602 XVs -> 798 XUVs
C:0 T:8 7456 XUs and 7176 XVs -> 74 XUVs
C:0 T:9 38202 XUs and 50176 XVs -> 2390 XUVs
C:0 T:10 16374 XUs and 9678 XVs -> 2291 XUVs
C:0 T:11 96 XUs and 42 XVs -> 18 XUVs
32495 XUVs total
10809 collection wire objects
28628 potential space points
Neighbour search...
3115808 tests to find 1369546 neighbours
Iterating with no regularization...
Begin: 5.71409e+07
0 4.45131e+07
1 4.37527e+07
2 4.36832e+07
3 4.36633e+07
Now with regularization...
Begin: 3.60023e+07
0 3.59801e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410635026
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410635026
fLifetime: 30000 [us]
20-Nov-2025 22:54:55 GMT  Closed output file "reco_stage1.root"
20-Nov-2025 22:54:55 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          266.191       417.278       606.778       401.578       100.907        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00123947    0.00363968    0.00578021    0.00365482    0.00138492       10     
reco:ophit:OpHitFinder                             0.234482      0.323096      0.507683      0.302988      0.0742425       10     
reco:opflash:OpFlashFinder                          0.10045      0.178937      0.269252      0.170808      0.0437667       10     
reco:opslicer:OpSlicer                             0.588349      0.828584       1.47899      0.732611      0.268688        10     
reco:caldata:DataPrepModule                         5.06033       5.92635       8.02482       5.54171      0.912913        10     
reco:wclsdatasp:WireCellToolkit                     34.6709       40.1315       46.0815       39.2883       4.2321         10     
reco:gaushit:GausHitFinder                          1.15734       1.45744       1.87237       1.3752       0.236929        10     
reco:nhitsfilter:NumberOfHitsFilter                0.0004678    0.000681236   0.000830748   0.000687956   0.000118165      10     
reco:reco3d:SpacePointSolver                        7.97709       19.417        33.6647       19.7764       7.76754        10     
reco:hitpdune:DisambigFromSpacePoints              0.761378       1.76786       2.74639       1.67967      0.632204        10     
reco:pandora:StandardPandora                        19.3344       44.6844       127.143       38.7107       29.2406        10     
reco:pandoraWriter:StandardPandora                 0.422433      0.520664      0.759708      0.491145      0.0932585       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.77891       4.9027        10.0685       4.77519       2.15159        10     
reco:pandoraShower:LArPandoraShowerCreation         2.24065       5.66928       10.1939       5.55243       2.65992        10     
reco:pandoracalo:Calorimetry                        1.2182        2.55537       4.56354       2.50651      0.870426        10     
reco:pandoracalonosce:Calorimetry                  0.908909       2.21657       3.60761       2.13324      0.749814        10     
reco:pandorapid:Chi2ParticleID                     0.0046455    0.00654214    0.00844177    0.00651863    0.00104495       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0399961     0.0610175     0.0926187     0.0553369     0.0142975       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00449972    0.00647988     0.0108577    0.00609048    0.00171426       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.66964       5.46058       9.59177       5.64771       2.4305         10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.02966       5.33808       9.58659       5.63576       2.24798        10     
reco:emtrkmichelid:EmTrackMichelId                  154.838       224.973       281.843       221.171       43.0669        10     
reco:crttag:SingleCRTMatchingProducer             0.00727735     0.0188112     0.0345485     0.0176319    0.00715846       10     
reco:crtreco:TwoCRTMatchingProducer               0.00555154     0.0120933     0.0175669     0.0135832    0.00428028       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.544255      0.972045       1.26312      0.959957      0.224951        10     
reco:pandora2Track:LArPandoraTrackCreation          5.42152       16.6819       28.0233       16.1985       7.16045        10     
reco:pandora2calo:Calorimetry                       2.69419       6.86974       12.4877       6.59744       2.75813        10     
reco:pandora2calonosce:Calorimetry                  2.32543       6.38818       10.8379       6.20292       2.52203        10     
reco:pandora2pid:Chi2ParticleID                    0.0103044     0.0162904     0.0226333     0.0158675    0.00387131       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0475929     0.0795989     0.182063      0.0716643     0.0365383       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0100015     0.0152709     0.0210048     0.0152425     0.0031965       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.12013       5.76258       9.54649       5.66212       2.28174        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.02436       5.60446       9.13959       5.28381       2.20053        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.87834       4.91533       7.42676       4.87874       1.75435        10     
[art]:TriggerResults:TriggerResultInserter        4.7221e-05    6.42493e-05   0.000145243   5.48165e-05   2.75846e-05      10     
end_path:out1:RootOutput                           9.061e-06    1.15492e-05    1.553e-05    1.1265e-05    2.08707e-06      10     
end_path:out1:RootOutput(write)                     2.20491       3.36534       4.30875       3.22153      0.704764        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5027.15 MB
  Peak resident set size usage (VmHWM): 4197.4 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.25 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.68 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9692260
-rw-r--r--. 1 pildune003 dunepil 2302151959 Nov 20 21:43 detsim_stage2.root
-rw-r--r--. 1 pildune003 dunepil 2186281234 Nov 20 21:40 detsim_stage1.root
-rw-r--r--. 1 pildune003 dunepil 1989650987 Nov 20 22:54 reco_stage1.root
-rw-r--r--. 1 pildune003 dunepil 1714449177 Nov 20 19:40 g4_stage2.root
-rw-r--r--. 1 pildune003 dunepil 1655924908 Nov 20 19:27 g4_stage1.root
-rw-r--r--. 1 pildune003 dunepil   75796510 Nov 20 22:49 Pandora_Events.pndr
-rw-r--r--. 1 pildune003 dunepil     243009 Nov 20 22:54 jobscript.log
-rw-r--r--. 1 pildune003 dunepil     217088 Nov 20 22:54 mem.db
-rw-r--r--. 1 pildune003 dunepil      62376 Nov 20 22:54 Pandora_Geometry.xml
-rw-r--r--. 1 pildune003 dunepil      36864 Nov 20 22:54 time.db
-rw-r--r--. 1 pildune003 dunepil        519 Nov 20 21:43 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 pildune003 dunepil        519 Nov 20 19:40 g4_protoDUNE_hist.root
-rw-r--r--. 1 pildune003 dunepil        519 Nov 20 22:54 protoDUNE_reco1_hist.root
-rw-r--r--. 1 pildune003 dunepil         52 Nov 20 19:19 all-input-dids.txt
-rw-r--r--. 1 pildune003 dunepil          7 Nov 20 22:54 justin-processed-pfns.txt
-rw-r--r--. 1 pildune003 dunepil          0 Nov 20 21:49 ProtoDUNE.log
-rw-r--r--. 1 pildune003 dunepil          0 Nov 20 21:43 debugprod.log
justIN time: 2026-02-09 03:28:13 UTC       justIN version: 01.06.00