justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 263520.0@dunegpschedd02.fnal.gov

Jobsub ID263520.0@dunegpschedd02.fnal.gov
Workflow ID11288
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-20 03:57:41
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce513
Last heartbeat2025-12-20 08:01:16
From worker nodeHostnameb9p06p6630.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-20 03:58:35
Input filesmonte-carlo-011288-000001
JobscriptExit code0
Real time4h (14462s)
CPU time3h (13760s = 95%)
Max RSS bytes4533882880 (4323 MiB)
Outputting started2025-12-20 07:59:39
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11288/1/001/reco_stage1.root
Finished2025-12-20 08:01:16
Saved logsjustin-logs:263520.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

leKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 140279 traces, 0 summary
Retagger: tagging trace set: gauss with 106836 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 106836 traces tagged "gauss"
FrameSaver: q=3.51456e+06 n=1872499 tag=gauss
wclsFrameSaver: saving 140279 traces tagged "wiener"
FrameSaver: q=3.72173e+06 n=1654175 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 25834 XUs and 26564 XVs -> 265 XUVs
C:0 T:1 39988 XUs and 38660 XVs -> 2433 XUVs
C:0 T:2 140903 XUs and 166392 XVs -> 10096 XUVs
C:0 T:3 58933 XUs and 56938 XVs -> 468 XUVs
C:0 T:4 11334 XUs and 12541 XVs -> 154 XUVs
C:0 T:5 42191 XUs and 36292 XVs -> 3320 XUVs
C:0 T:6 171331 XUs and 190644 XVs -> 18971 XUVs
C:0 T:7 57004 XUs and 60406 XVs -> 848 XUVs
C:0 T:8 8296 XUs and 8820 XVs -> 112 XUVs
C:0 T:9 49267 XUs and 72659 XVs -> 4023 XUVs
C:0 T:10 25545 XUs and 17534 XVs -> 4806 XUVs
C:0 T:11 216 XUs and 144 XVs -> 18 XUVs
45514 XUVs total
14153 collection wire objects
41982 potential space points
Neighbour search...
4961604 tests to find 2531476 neighbours
Iterating with no regularization...
Begin: 5.33061e+07
0 4.46599e+07
1 4.39526e+07
2 4.3853e+07
3 4.38255e+07
Now with regularization...
Begin: 3.42504e+07
0 3.42105e+07
1 3.41756e+07
2 3.41497e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411226047
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411226047
fLifetime: 30000 [us]
20-Dec-2025 08:59:36 CET  Closed output file "reco_stage1.root"
20-Dec-2025 08:59:36 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          278.162       420.681       633.307       381.361       110.422        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00123171    0.00430665    0.00701815    0.00457971    0.00188167       10     
reco:ophit:OpHitFinder                             0.240781       0.36715      0.496978      0.370333      0.084815        10     
reco:opflash:OpFlashFinder                         0.154701      0.230537       0.37191      0.222828      0.0606292       10     
reco:opslicer:OpSlicer                             0.504462      0.831112       1.21064      0.852161      0.222168        10     
reco:caldata:DataPrepModule                         4.56614       6.13707       7.22222       6.22021      0.770233        10     
reco:wclsdatasp:WireCellToolkit                     35.8808       39.201        43.0749       38.7458       2.25747        10     
reco:gaushit:GausHitFinder                         0.905532       1.3889        2.12756       1.30866      0.401845        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000426178   0.000611066   0.000935496   0.000543904   0.000173774      10     
reco:reco3d:SpacePointSolver                        9.32395       20.5925       42.6196       16.8325       9.56933        10     
reco:hitpdune:DisambigFromSpacePoints               1.19305       2.06617       3.56627       1.93408      0.667231        10     
reco:pandora:StandardPandora                        23.5976       43.9675       98.5761       33.5344       23.8245        10     
reco:pandoraWriter:StandardPandora                 0.365902       0.54751      0.848106      0.507911      0.137605        10     
reco:pandoraTrack:LArPandoraTrackCreation           2.28477       4.67952       7.81326       4.22057       1.58657        10     
reco:pandoraShower:LArPandoraShowerCreation         2.36613       5.12324       9.58154       4.26542       2.10325        10     
reco:pandoracalo:Calorimetry                        1.3469        2.93959       4.98386       2.4667        1.13424        10     
reco:pandoracalonosce:Calorimetry                   1.18842       2.44067       4.45532       2.25166      0.965332        10     
reco:pandorapid:Chi2ParticleID                    0.00397897    0.00728634     0.0126033    0.00721791    0.00249908       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0392897     0.0737634     0.104344      0.0764779     0.0213802       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00375373    0.00728903     0.0108394    0.00753649     0.0022478       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.41771       5.45108       10.3931       4.61508       2.59025        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.33816       5.56035       10.3555       4.60958       2.52095        10     
reco:emtrkmichelid:EmTrackMichelId                  166.107       229.864       318.66        213.857       48.6939        10     
reco:crttag:SingleCRTMatchingProducer              0.0114449     0.0180108     0.0258567     0.0185885    0.00448662       10     
reco:crtreco:TwoCRTMatchingProducer               0.00607666     0.0106848     0.0168638     0.0109178     0.0029458       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.698185       1.1042        1.66461       1.02203      0.271793        10     
reco:pandora2Track:LArPandoraTrackCreation          6.30445       13.7793       25.7366       12.1401       5.69147        10     
reco:pandora2calo:Calorimetry                       3.28786       7.6406        14.3652       6.50703       3.31355        10     
reco:pandora2calonosce:Calorimetry                  2.4629        7.12519       13.4119       6.18324       3.29926        10     
reco:pandora2pid:Chi2ParticleID                   0.00812767     0.0146636     0.0281972     0.0111504    0.00645762       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0383466     0.0716429     0.123743      0.0561935     0.028521        10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00774831     0.0143313     0.0269366     0.0118995    0.00603261       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.82294       5.05864       9.80044       4.42935       2.45639        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.35941       6.07763       11.0132       5.29558       2.60341        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.93033       5.06122       8.3573        4.72841       1.85948        10     
[art]:TriggerResults:TriggerResultInserter        4.7201e-05    6.09649e-05   0.000139832   5.26655e-05   2.65873e-05      10     
end_path:out1:RootOutput                           8.48e-06     9.7481e-06    1.5291e-05     9.205e-06    1.87589e-06      10     
end_path:out1:RootOutput(write)                     1.78034       3.11138       4.44123       3.0293       0.701102        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5377.62 MB
  Peak resident set size usage (VmHWM): 4533.88 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.27 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.75 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11381612
-rw-r--r--. 1 duneprd np-comp 2630426593 Dec 20 07:46 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2495695630 Dec 20 07:44 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2353256352 Dec 20 08:59 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 2089072540 Dec 20 05:22 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 2001886145 Dec 20 05:05 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   83824640 Dec 20 08:54 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     243159 Dec 20 08:59 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Dec 20 08:59 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Dec 20 08:59 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Dec 20 08:59 time.db
-rw-r--r--. 1 duneprd np-comp        519 Dec 20 07:46 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Dec 20 05:22 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Dec 20 08:59 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Dec 20 04:58 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Dec 20 08:59 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Dec 20 07:53 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Dec 20 07:46 debugprod.log
justIN time: 2025-12-20 18:37:07 UTC       justIN version: 01.05.03