justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 251505.1@dunegpschedd01.fnal.gov

Jobsub ID251505.1@dunegpschedd01.fnal.gov
Workflow ID10214
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-15 08:06:33
SiteIT_CNAF
EntryCMSHTPC_T1_IT_CNAF_condor_ce02
Last heartbeat2025-11-15 13:06:07
From worker nodeHostnamewn-f01-16-03-10.cr.cnaf.infn.it
cpuinfoIntel(R) Xeon(R) CPU E5-2697 v4 @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-15 08:07:33
Input filesmonte-carlo-010214-000001
JobscriptExit code0
Real time4h (17849s)
CPU time4h (17447s = 97%)
Max RSS bytes4438380544 (4232 MiB)
Outputting started2025-11-15 13:05:02
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10214/1/001/reco_stage1.root
Finished2025-11-15 13:06:07
Saved logsjustin-logs:251505.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 120548 traces, 0 summary
Retagger: tagging trace set: gauss with 97445 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 97445 traces tagged "gauss"
FrameSaver: q=3.44873e+06 n=1804277 tag=gauss
wclsFrameSaver: saving 120548 traces tagged "wiener"
FrameSaver: q=3.65281e+06 n=1641531 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 20398 XUs and 27878 XVs -> 208 XUVs
C:0 T:1 36672 XUs and 40406 XVs -> 2794 XUVs
C:0 T:2 110486 XUs and 67368 XVs -> 6324 XUVs
C:0 T:3 36691 XUs and 26782 XVs -> 213 XUVs
C:0 T:4 17443 XUs and 18767 XVs -> 321 XUVs
C:0 T:5 60744 XUs and 51610 XVs -> 3915 XUVs
C:0 T:6 142154 XUs and 129907 XVs -> 5599 XUVs
C:0 T:7 60942 XUs and 53709 XVs -> 534 XUVs
C:0 T:8 9756 XUs and 13468 XVs -> 200 XUVs
C:0 T:9 60403 XUs and 96453 XVs -> 3935 XUVs
C:0 T:10 8808 XUs and 5162 XVs -> 926 XUVs
C:0 T:11 118 XUs and 71 XVs -> 11 XUVs
24980 XUVs total
10217 collection wire objects
22828 potential space points
Neighbour search...
937262 tests to find 472148 neighbours
Iterating with no regularization...
Begin: 8.42712e+07
0 6.8482e+07
1 6.75053e+07
2 6.73838e+07
3 6.73499e+07
Now with regularization...
Begin: 5.73788e+07
0 5.73544e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410525495
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410525495
fLifetime: 30000 [us]
15-Nov-2025 14:05:00 CET  Closed output file "reco_stage1.root"
15-Nov-2025 14:05:00 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          602.763       718.006       985.573       703.268       115.11         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00129109    0.00409154    0.00827782    0.00361865    0.00216154       10     
reco:ophit:OpHitFinder                             0.290058      0.417715      0.525257      0.411732      0.0606192       10     
reco:opflash:OpFlashFinder                         0.149617      0.205477      0.314565       0.20289      0.0426208       10     
reco:opslicer:OpSlicer                             0.678957       1.12483       1.3628        1.15386      0.196557        10     
reco:caldata:DataPrepModule                         5.75057       9.0261        9.75117       9.37717       1.11745        10     
reco:wclsdatasp:WireCellToolkit                     50.0438       62.0585       69.4064       61.3204       5.42711        10     
reco:gaushit:GausHitFinder                          1.71307       2.41901       2.93699       2.47141       0.35163        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000713657   0.000871471   0.00100862    0.000851674   8.73901e-05      10     
reco:reco3d:SpacePointSolver                        16.2776       24.6482       43.3187       22.5801       8.02519        10     
reco:hitpdune:DisambigFromSpacePoints               3.06257       4.33948       5.35473       4.42615      0.695747        10     
reco:pandora:StandardPandora                        44.7728       69.1515       97.6778       64.0319       16.4423        10     
reco:pandoraWriter:StandardPandora                 0.534981      0.696447      0.859848      0.716715      0.117084        10     
reco:pandoraTrack:LArPandoraTrackCreation           7.94039       10.7306       14.4279       10.0457       2.0473         10     
reco:pandoraShower:LArPandoraShowerCreation         9.71847       12.6567       20.8456       11.536        3.14591        10     
reco:pandoracalo:Calorimetry                        3.92412       5.16501       6.43233       4.97007      0.885816        10     
reco:pandoracalonosce:Calorimetry                   2.97088       4.72869       6.15482       4.51814      0.946512        10     
reco:pandorapid:Chi2ParticleID                    0.00646168    0.00925332     0.0118324    0.00896658    0.00202688       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0735777     0.0982716     0.128825      0.101735      0.0162563       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00646582    0.00899146     0.0119319    0.00874135    0.00192124       10     
reco:pandoraShowercalo:ShowerCalorimetry            7.27724       12.2182       17.821        11.5862       2.95926        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       7.09763       12.0341       17.4667       12.1383       3.04524        10     
reco:emtrkmichelid:EmTrackMichelId                  287.788       353.81        457.308       348.983       54.5454        10     
reco:crttag:SingleCRTMatchingProducer              0.0251645     0.0333424     0.0423436     0.0329046     0.005106        10     
reco:crtreco:TwoCRTMatchingProducer                0.0201798     0.0260276     0.034942      0.0251679    0.00387602       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.69816       2.37227       3.55468       2.36514      0.493592        10     
reco:pandora2Track:LArPandoraTrackCreation          22.4518       44.7251       82.8424       41.4723       15.837         10     
reco:pandora2calo:Calorimetry                       9.33628       17.6894       31.1154       16.7171       6.06343        10     
reco:pandora2calonosce:Calorimetry                  8.6415        16.7254       31.5426       15.3731       6.20224        10     
reco:pandora2pid:Chi2ParticleID                    0.0175926     0.0297374     0.0453035     0.0308355    0.00809238       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0957559     0.147663      0.214805      0.143985      0.0334642       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.017613      0.0293752     0.0453509     0.0291591    0.00795653       10     
reco:pandora2Shower:LArPandoraShowerCreation        6.56848       13.7256       25.3824       13.317        4.99255        10     
reco:pandora2Showercalo:ShowerCalorimetry           7.73344       16.7612       29.1309       16.8415       5.62233        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      6.99988       15.1362       28.2391       14.9067       5.42959        10     
[art]:TriggerResults:TriggerResultInserter        5.6384e-05    8.86482e-05   0.000173771   8.1863e-05    3.15067e-05      10     
end_path:out1:RootOutput                           9.093e-06    1.28325e-05   2.1031e-05    1.25225e-05   3.00836e-06      10     
end_path:out1:RootOutput(write)                     2.74354       4.91415       7.32724       5.00421       1.19564        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5244.99 MB
  Peak resident set size usage (VmHWM): 4438.38 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.36 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.75 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.30 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10426724
-rw-r--r-- 1 neutrino002 neutrino 2449116841 Nov 15 12:02 detsim_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 2330776642 Nov 15 11:58 detsim_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2170705718 Nov 15 14:05 reco_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 1857734203 Nov 15 09:34 g4_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 1784610618 Nov 15 09:18 g4_stage1.root
-rw-r--r-- 1 neutrino002 neutrino   83390470 Nov 15 13:58 Pandora_Events.pndr
-rw-r--r-- 1 neutrino002 neutrino     242863 Nov 15 14:05 jobscript.log
-rw-r--r-- 1 neutrino002 neutrino     217088 Nov 15 14:05 mem.db
-rw-r--r-- 1 neutrino002 neutrino      62376 Nov 15 14:05 Pandora_Geometry.xml
-rw-r--r-- 1 neutrino002 neutrino      36864 Nov 15 14:05 time.db
-rw-r--r-- 1 neutrino002 neutrino        519 Nov 15 12:02 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Nov 15 09:34 g4_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Nov 15 14:05 protoDUNE_reco1_hist.root
-rw-r--r-- 1 neutrino002 neutrino         52 Nov 15 09:07 all-input-dids.txt
-rw-r--r-- 1 neutrino002 neutrino          7 Nov 15 14:05 justin-processed-pfns.txt
-rw-r--r-- 1 neutrino002 neutrino          0 Nov 15 12:11 ProtoDUNE.log
-rw-r--r-- 1 neutrino002 neutrino          0 Nov 15 12:02 debugprod.log
justIN time: 2026-02-09 12:25:49 UTC       justIN version: 01.06.00