justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 241704.0@dunegpschedd01.fnal.gov

Jobsub ID241704.0@dunegpschedd01.fnal.gov
Workflow ID9514
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-01 16:22:27
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2025-11-02 00:19:31
From worker nodeHostnamewn1205091.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-01 16:23:15
Input filesmonte-carlo-009514-000001
JobscriptExit code0
Real time7h (28456s)
CPU time7h (27491s = 96%)
Max RSS bytes5086892032 (4851 MiB)
Outputting started2025-11-02 00:17:33
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09514/1/001/reco_stage1.root
Finished2025-11-02 00:19:31
Saved logsjustin-logs:241704.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

p_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 204585 traces, 0 summary
Retagger: tagging trace set: gauss with 159819 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 159819 traces tagged "gauss"
FrameSaver: q=6.04762e+06 n=2975921 tag=gauss
wclsFrameSaver: saving 204585 traces tagged "wiener"
FrameSaver: q=6.46277e+06 n=2681611 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 50154 XUs and 44711 XVs -> 735 XUVs
C:0 T:1 129381 XUs and 138950 XVs -> 10792 XUVs
C:0 T:2 250538 XUs and 291779 XVs -> 15848 XUVs
C:0 T:3 92582 XUs and 90639 XVs -> 1143 XUVs
C:0 T:4 19293 XUs and 18037 XVs -> 338 XUVs
C:0 T:5 106448 XUs and 142688 XVs -> 13766 XUVs
C:0 T:6 205440 XUs and 245274 XVs -> 8625 XUVs
C:0 T:7 74419 XUs and 87244 XVs -> 1021 XUVs
C:0 T:8 8910 XUs and 8989 XVs -> 243 XUVs
C:0 T:9 75544 XUs and 82168 XVs -> 6142 XUVs
C:0 T:10 286102 XUs and 220984 XVs -> 25640 XUVs
C:0 T:11 740 XUs and 446 XVs -> 37 XUVs
84330 XUVs total
23281 collection wire objects
76180 potential space points
Neighbour search...
11359010 tests to find 3854504 neighbours
Iterating with no regularization...
Begin: 2.72566e+08
0 1.49172e+08
1 1.38843e+08
2 1.37555e+08
3 1.3719e+08
4 1.37027e+08
5 1.3694e+08
Now with regularization...
Begin: 1.10338e+08
0 1.09862e+08
1 1.09671e+08
2 1.09565e+08
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410250811
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410250811
fLifetime: 30000 [us]
02-Nov-2025 00:17:29 GMT  Closed output file "reco_stage1.root"
02-Nov-2025 00:17:29 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                           615.4        1007.01       2049.34       937.61        395.709        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00187823     0.0056098    0.00998182    0.00502419    0.00274426       10     
reco:ophit:OpHitFinder                              0.4576       0.573412      0.675488      0.585109      0.0725278       10     
reco:opflash:OpFlashFinder                         0.302801      0.403116      0.546064      0.390369      0.0718094       10     
reco:opslicer:OpSlicer                              1.31806       1.63484       2.09371       1.66287      0.215847        10     
reco:caldata:DataPrepModule                         8.72295       9.40341       10.0481       9.42627      0.356047        10     
reco:wclsdatasp:WireCellToolkit                     59.9853       64.1337       69.2012       63.9059       3.25399        10     
reco:gaushit:GausHitFinder                          1.9084        2.72027       4.23882       2.51602      0.698577        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000989452   0.00142684    0.00210263    0.00120249    0.000390455      10     
reco:reco3d:SpacePointSolver                        17.4249       37.5808       89.1807       31.9642       20.6595        10     
reco:hitpdune:DisambigFromSpacePoints               2.25626       4.60665       8.43678       4.4376        1.74336        10     
reco:pandora:StandardPandora                        46.8326       162.335       657.139       109.692       171.549        10     
reco:pandoraWriter:StandardPandora                 0.794169       1.08282       1.68909      0.992631      0.264989        10     
reco:pandoraTrack:LArPandoraTrackCreation           8.43821       15.2357       31.9411       14.2778       6.85091        10     
reco:pandoraShower:LArPandoraShowerCreation         7.5253        19.2538       44.2145       17.3761       10.4176        10     
reco:pandoracalo:Calorimetry                        3.86155       8.55392       17.3047       8.00466       3.72721        10     
reco:pandoracalonosce:Calorimetry                   3.70329       7.82531       15.332        6.64895       3.45251        10     
reco:pandorapid:Chi2ParticleID                    0.00900243     0.0131812     0.0210697     0.0115597    0.00374695       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0843351     0.118169      0.178572      0.109694      0.0318019       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00899285     0.0128645     0.0207528     0.0118942    0.00331545       10     
reco:pandoraShowercalo:ShowerCalorimetry            6.26153       19.4699       38.3897       17.8748       8.71385        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       8.22594       19.8401       45.2139       19.2805       10.3178        10     
reco:emtrkmichelid:EmTrackMichelId                  346.953       474.567       699.575       466.218       104.284        10     
reco:crttag:SingleCRTMatchingProducer              0.0212332     0.0325697     0.0468412      0.02942     0.00980118       10     
reco:crtreco:TwoCRTMatchingProducer                0.013587      0.0254593     0.0371979     0.0265582    0.00734733       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.37587       1.82132       2.50854       1.77934      0.353963        10     
reco:pandora2Track:LArPandoraTrackCreation          22.6432       47.9744       98.7439       43.6185       22.1681        10     
reco:pandora2calo:Calorimetry                       10.1082       26.2957       61.8562       21.7705       14.0044        10     
reco:pandora2calonosce:Calorimetry                  11.4095       24.0406       53.0333       21.0878       11.0655        10     
reco:pandora2pid:Chi2ParticleID                    0.0271639     0.0330358     0.0541693     0.029681     0.00839465       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.114314      0.153646      0.245219      0.140927      0.0391694       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0262975     0.0347808     0.0531834     0.0310528    0.00847166       10     
reco:pandora2Shower:LArPandoraShowerCreation        9.06684       15.0834       25.281        14.2701       4.78028        10     
reco:pandora2Showercalo:ShowerCalorimetry           8.46985       19.3313       36.252        18.5037       8.00688        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      7.27916       16.6281       30.281        16.2098       6.81497        10     
[art]:TriggerResults:TriggerResultInserter        4.0516e-05    6.74246e-05   0.000177394   5.1166e-05    3.95374e-05      10     
end_path:out1:RootOutput                           6.693e-06    8.1345e-06    1.5368e-05    7.4245e-06    2.44035e-06      10     
end_path:out1:RootOutput(write)                      3.789        5.91995       7.45431       6.09262       1.21827        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5926.87 MB
  Peak resident set size usage (VmHWM): 5086.89 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.39 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   6.10 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.24 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 12727756
-rw-r--r-- 1 dune01 dune01 2907662148 Nov  1 21:25 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2760116194 Nov  1 21:19 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2663453389 Nov  2 00:17 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 2353760026 Nov  1 17:08 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 2254780176 Nov  1 16:35 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   92819120 Nov  1 23:57 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     243235 Nov  2 00:17 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Nov  2 00:17 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Nov  2 00:17 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Nov  2 00:17 time.db
-rw-r--r-- 1 dune01 dune01        519 Nov  1 21:25 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov  1 17:08 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov  2 00:17 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Nov  1 16:23 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Nov  2 00:17 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Nov  1 21:43 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Nov  1 21:25 debugprod.log
justIN time: 2025-11-05 04:49:56 UTC       justIN version: 01.05.01