justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 19786.0@dunegpschedd01.fnal.gov

Jobsub ID19786.0@dunegpschedd01.fnal.gov
Workflow ID172
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-07-31 11:04:47
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce509
Last heartbeat2025-07-31 14:47:45
From worker nodeHostnameb9p12p6813.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-07-31 11:05:54
Input filesmonte-carlo-000172-000001
JobscriptExit code0
Real time3h (13202s)
CPU time3h (12281s = 93%)
Max RSS bytes4410544128 (4206 MiB)
Outputting started2025-07-31 14:45:57
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/00172/1/001/reco_stage1.root
Finished2025-07-31 14:47:45
Saved logsjustin-logs:19786.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

dcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 117710 traces, 0 summary
Retagger: tagging trace set: gauss with 90579 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 90579 traces tagged "gauss"
FrameSaver: q=3.34725e+06 n=1650522 tag=gauss
wclsFrameSaver: saving 117710 traces tagged "wiener"
FrameSaver: q=3.52915e+06 n=1474822 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 56468 XUs and 51342 XVs -> 860 XUVs
C:0 T:1 124908 XUs and 106244 XVs -> 9088 XUVs
C:0 T:2 42381 XUs and 65502 XVs -> 5148 XUVs
C:0 T:4 5147 XUs and 7476 XVs -> 54 XUVs
C:0 T:5 16710 XUs and 21059 XVs -> 2839 XUVs
C:0 T:6 93481 XUs and 96800 XVs -> 3294 XUVs
C:0 T:7 42903 XUs and 39563 XVs -> 288 XUVs
C:0 T:8 5784 XUs and 5871 XVs -> 52 XUVs
C:0 T:9 30760 XUs and 44847 XVs -> 2839 XUVs
C:0 T:10 49982 XUs and 40894 XVs -> 5128 XUVs
C:0 T:11 357 XUs and 253 XVs -> 37 XUVs
29627 XUVs total
10066 collection wire objects
26071 potential space points
Neighbour search...
1546977 tests to find 867596 neighbours
Iterating with no regularization...
Begin: 6.75449e+07
0 5.8354e+07
1 5.76456e+07
2 5.75327e+07
3 5.75006e+07
Now with regularization...
Begin: 4.73123e+07
0 4.72868e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 408352886
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 408352886
fLifetime: 30000 [us]
31-Jul-2025 16:45:55 CEST  Closed output file "reco_stage1.root"
31-Jul-2025 16:45:55 CEST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          276.415       353.253       498.21        325.958       70.7565        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00122374    0.00351886    0.00613831    0.00366977    0.00146311       10     
reco:ophit:OpHitFinder                             0.225205      0.273194      0.360225      0.269671      0.0352831       10     
reco:opflash:OpFlashFinder                         0.148089      0.177129      0.232706      0.164209      0.0280834       10     
reco:opslicer:OpSlicer                             0.475546      0.649173      0.825434      0.627891      0.126648        10     
reco:caldata:DataPrepModule                         4.5728        5.47091       7.09753       5.16607       0.77623        10     
reco:wclsdatasp:WireCellToolkit                     30.7552       33.5353       35.9766       33.7101       1.80289        10     
reco:gaushit:GausHitFinder                         0.876384       1.10899       1.34989       1.10659      0.156348        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000430614   0.000579661   0.000817217   0.00054316    0.000111724      10     
reco:reco3d:SpacePointSolver                        9.8433         17.41        29.0063       14.9655       6.92435        10     
reco:hitpdune:DisambigFromSpacePoints               1.30118       1.80891       2.9586        1.43325      0.598231        10     
reco:pandora:StandardPandora                        21.7107       34.8676       78.4223       25.4367       17.1406        10     
reco:pandoraWriter:StandardPandora                 0.274753      0.429224      0.667548      0.339205      0.146858        10     
reco:pandoraTrack:LArPandoraTrackCreation           2.35638       3.88531       6.69478       3.10977       1.35589        10     
reco:pandoraShower:LArPandoraShowerCreation         2.63019       4.21321       6.72141       3.86599       1.32043        10     
reco:pandoracalo:Calorimetry                        1.30194       2.29011       3.31169       1.83645      0.773607        10     
reco:pandoracalonosce:Calorimetry                   1.25086       2.19726       3.81508       1.69612      0.853884        10     
reco:pandorapid:Chi2ParticleID                     0.0039506    0.00605126    0.00908563    0.00513148    0.00216227       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0309153     0.0559529     0.0899257     0.0494086     0.0211503       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00378597    0.00582256    0.00904391    0.00544745    0.00188424       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.79116       4.7688        7.70369       4.78967       1.5577         10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.86587       4.73201       7.01624       4.65318       1.47452        10     
reco:emtrkmichelid:EmTrackMichelId                  152.337       191.855       250.885       182.278       30.4984        10     
reco:crttag:SingleCRTMatchingProducer             0.00935168     0.0162425     0.0238178     0.016238     0.00477863       10     
reco:crtreco:TwoCRTMatchingProducer                0.0055087    0.00846496     0.0120212    0.00843551    0.00198815       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.749753       0.99372       1.21601      0.974348      0.143182        10     
reco:pandora2Track:LArPandoraTrackCreation          7.58813       11.9804       19.9172       10.7962       3.96509        10     
reco:pandora2calo:Calorimetry                       4.36482       6.55075       9.39839       6.22101       1.6571         10     
reco:pandora2calonosce:Calorimetry                  4.27338       6.18363       9.94846       5.47861       1.70918        10     
reco:pandora2pid:Chi2ParticleID                   0.00879504     0.0153529     0.0212053     0.0153905    0.00433238       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.047002      0.0753651     0.103378      0.0752617     0.0195824       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00982826     0.0152506     0.0205954     0.0155087    0.00333969       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.33416       4.51338       6.33798       3.98637       1.01354        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.74211       5.5806        7.82858       5.24807       1.36783        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.41256       4.66289       6.82068       4.34403       1.12919        10     
[art]:TriggerResults:TriggerResultInserter        4.6731e-05    6.75406e-05   0.000164942   5.60155e-05   3.32205e-05      10     
end_path:out1:RootOutput                           8.15e-06     1.0192e-05     1.571e-05     9.025e-06    2.61123e-06      10     
end_path:out1:RootOutput(write)                     2.16021       2.79897       3.56704       2.75418      0.441818        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5242.88 MB
  Peak resident set size usage (VmHWM): 4410.54 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.24 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.44 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10834364
-rw-r--r--. 1 duneprd np-comp 2517991159 Jul 31 15:43 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2393193257 Jul 31 15:41 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2214932809 Jul 31 16:45 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1981461584 Jul 31 13:31 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1908292147 Jul 31 13:14 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   77902020 Jul 31 16:42 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     243205 Jul 31 16:45 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jul 31 16:45 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jul 31 16:45 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jul 31 16:45 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jul 31 15:43 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jul 31 13:31 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jul 31 16:45 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jul 31 13:06 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jul 31 16:45 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jul 31 15:50 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jul 31 15:43 debugprod.log
justIN time: 2025-08-04 22:46:20 UTC       justIN version: 01.04.00