justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 242591.1@dunegpschedd01.fnal.gov

Jobsub ID242591.1@dunegpschedd01.fnal.gov
Workflow ID9566
Stage ID1
User namemwrobel@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes9437184000 (9000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-11-03 19:57:13
SiteUS_Wisconsin
EntryHCCHTPC_US_Wisconsin_osg01_rhel7
Last heartbeat2025-11-03 21:40:05
From worker nodeHostnamee2590
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit82800 (23 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-03 20:02:20
Input filesusertests:pdhd_prod_beam__231902_43_1_20251008T212443Z_gen_g4_IonScintPDExt.root_227040_85_1_20251010T170521Z_PDInt.root
JobscriptExit code0
Real time1h (5637s)
CPU time1h (5228s = 92%)
Max RSS bytes6555488256 (6251 MiB)
Outputting started2025-11-03 21:36:18
Output filesusertests:pdhd_prod_beam__231902_43_1_20251008T212443Z_gen_g4_IonScintPDExt.root_227040_85_1_20251010T170521Z_PDInt.root_242591_1_1_20251103T200229Z_detsim_reco1.root
Finished2025-11-03 21:40:05
Saved logsjustin-logs:242591.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

205344 neighbours
Iterating with no regularization...
Begin: 6.11498e+07
0 5.37276e+07
1 5.3174e+07
2 5.31112e+07
3 5.30963e+07
Now with regularization...
Begin: 5.10736e+07
0 5.10627e+07
Begin processing the 10th record. run: 20250627 subRun: 1 event: 99 at 03-Nov-2025 15:32:39 CST

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 16137!
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 92868 traces, 92868 summary
Retagger: tagging trace set: gauss with 76690 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 76690 traces tagged "gauss"
FrameSaver: q=2.2756e+06 n=1172637 tag=gauss
wclsFrameSaver: saving 92868 traces tagged "wiener"
FrameSaver: q=3.14207e+06 n=1157143 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 17693 XUs and 11980 XVs -> 685 XUVs
C:0 T:1 228989 XUs and 149729 XVs -> 9623 XUVs
C:0 T:2 26412 XUs and 38990 XVs -> 653 XUVs
C:0 T:5 22520 XUs and 13887 XVs -> 433 XUVs
C:0 T:6 11886 XUs and 23267 XVs -> 163 XUVs
11557 XUVs total
3037 collection wire objects
11557 potential space points
Neighbour search...
815713 tests to find 279854 neighbours
Iterating with no regularization...
Begin: 4.39682e+07
0 3.56998e+07
1 3.48818e+07
2 3.47806e+07
3 3.47518e+07
Now with regularization...
Begin: 3.36999e+07
0 3.36821e+07
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.00279391 0.00208658 0.00124419 0.0134271 0.020591 0.0222852 0.0214036 0.00606596 0.00386505 0.00501175 0.00536263 
----view 1: 0.00171685 0.0017544 0.00178266 0.00185347 0.00182498 0.00175261 0.00162786 0.00177413 0.0018512 0.00189364 0.00192845 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.00208658 0.00124419 0.0134271 0.020591 0.0222852 0.0214036 0.00606596 0.00386505 0.00501175 0.00536263 0.00309467 
----view 1: 0.0017544 0.00178266 0.00185347 0.00182498 0.00175261 0.00162786 0.00177413 0.0018512 0.00189364 0.00192845 0.00169504 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.00124419 0.0134271 0.020591 0.0222852 0.0214036 0.00606596 0.00386505 0.00501175 0.00536263 0.00309467 0.00732109 
----view 1: 0.00178266 0.00185347 0.00182498 0.00175261 0.00162786 0.00177413 0.0018512 0.00189364 0.00192845 0.00169504 0.0016436 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.0134271 0.020591 0.0222852 0.0214036 0.00606596 0.00386505 0.00501175 0.00536263 0.00309467 0.00732109 0.00187936 
----view 1: 0.00185347 0.00182498 0.00175261 0.00162786 0.00177413 0.0018512 0.00189364 0.00192845 0.00169504 0.0016436 0.00164419 
03-Nov-2025 15:36:16 CST  Closed output file "pdhd_prod_beam__231902_43_1_20251008T212443Z_gen_g4_IonScintPDExt.root_227040_85_1_20251010T170521Z_PDInt.root_242591_1_1_20251103T200229Z_detsim_reco1.root"
03-Nov-2025 15:36:16 CST  Closed input file "temp_detsim.root"

======================================================================================================================================
TimeTracker printout (sec)                              Min           Avg           Max         Median          RMS         nEvts   
======================================================================================================================================
Full event                                            120.567       183.568       227.932       190.388       36.6497        10     
--------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                              0.00105219    0.00774887     0.0174283    0.00728323    0.00476975       10     
reco:opdec:Deconvolution                             0.625774      0.869953       1.06039      0.882438      0.143377        10     
reco:ophitspe:OpHitFinderDeco                        0.142472      0.218838      0.269617      0.226672      0.0374505       10     
reco:opflash:OpFlashFinder                           0.064928      0.0985665     0.141766      0.0983447     0.0228656       10     
reco:opslicer:OpSlicer                               0.152656       0.26593       0.35633      0.252514      0.0643416       10     
reco:wclsdatahd:WireCellToolkit                       54.407        58.3188       69.632        57.1011       4.31478        10     
reco:gaushit:GausHitFinder                           0.562633      0.928956       1.35048      0.869136      0.216584        10     
reco:nhitsfilter:NumberOfHitsFilter                 0.000509906   0.00081659    0.00113521    0.00079705    0.000232595      10     
reco:reco3d:SpacePointSolver                         0.424728       1.66978       4.32074       1.37723       1.08139        10     
reco:hitpdune:DisambigFromSpacePoints                 0.34798      0.745215       1.09045       0.8516       0.248429        10     
reco:pandora:StandardPandora                          12.5138       34.2468       67.3116       29.8979       13.7725        10     
reco:pandoraWriter:StandardPandora                   0.314647      0.473788      0.768362      0.488701      0.124378        10     
reco:pandoraTrack:LArPandoraTrackCreation             1.57175       3.00815       5.46591       2.56066       1.30117        10     
reco:pandoraShower:LArPandoraShowerCreation           1.06844       2.59359       4.65665       2.32953       1.16933        10     
reco:pandoracalo:Calorimetry                         0.753167       1.39172       2.39441       1.39534      0.510983        10     
reco:pandoracalonosce:Calorimetry                    0.692145       1.40114       2.44621       1.32054      0.576388        10     
reco:pandoraShowercalo:ShowerCalorimetry              1.02282       2.3986        4.33812       2.35468       1.02778        10     
reco:pandoraShowercalonosce:ShowerCalorimetry        0.975856       2.41239       4.83694       2.32654       1.14918        10     
reco:pandoraGnocchiCalo:GnocchiCalorimetry           0.0379171     0.0686999     0.110924      0.0622074     0.0231424       10     
reco:pandoraGnocchiCalonosce:GnocchiCalorimetry      0.0407368     0.0703458     0.108791      0.0663171     0.0227894       10     
reco:emtrkmichelid:EmTrackMichelId                    28.7358       45.4128       59.1299       47.2621       8.81477        10     
reco:crttag:SingleCRTMatchingProducer               0.00705676     0.0163972     0.0285619     0.0151398    0.00611792       10     
reco:crtreco:TwoCRTMatchingProducer                 0.00660831     0.0119191     0.0259286    0.00827795    0.00655477       10     
reco:pandora2Track:LArPandoraTrackCreation            3.10729       8.93104       18.0097       7.93271       5.09647        10     
reco:pandora2calo:ShowerCalorimetry                  0.916558       2.39095       5.51171       2.21456       1.41228        10     
reco:pandora2calonosce:ShowerCalorimetry             0.944517       2.30783       4.52482       1.76832       1.31851        10     
reco:pandora2GnocchiCalo:GnocchiCalorimetry          0.0346223     0.0653248     0.100449      0.064507      0.0181027       10     
reco:pandora2GnocchiCalonosce:GnocchiCalorimetry     0.0465504     0.0754325     0.108636      0.0696305     0.0226483       10     
reco:pandora2Shower:LArPandoraShowerCreation          1.3344        3.61114       9.03429       2.96149       2.54248        10     
reco:pandora2Showercalo:ShowerCalorimetry             1.20336       3.61182       7.07666       3.15122       2.14372        10     
reco:pandora2Showercalonosce:ShowerCalorimetry        1.13358       3.03278       5.80076       2.80657       1.63351        10     
[art]:TriggerResults:TriggerResultInserter          3.6389e-05    0.000129733   0.000519925   5.43475e-05   0.000148271      10     
end_path:out1:RootOutput                            1.2524e-05    1.82105e-05   5.0004e-05    1.4447e-05    1.07505e-05      10     
end_path:out1:RootOutput(write)                       2.09442       2.6699        3.33309       2.66357      0.342442        10     
======================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 6080.67 MB
  Peak resident set size usage (VmHWM): 3805.64 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Reco returns 0
.:
total 11181916
-rw-r--r-- 1 slot1_17 slot1_17      47827 Nov  3 15:36 jobscript.log
-rw-r--r-- 1 slot1_17 slot1_17        200 Nov  3 15:36 justin-processed-pfns.txt
-rw-r--r-- 1 slot1_17 slot1_17        519 Nov  3 15:36 reco_protodunehd_hist.root
-rw-r--r-- 1 slot1_17 slot1_17       3574 Nov  3 15:36 Pandora_Geometry.xml
-rw-r--r-- 1 slot1_17 slot1_17     200704 Nov  3 15:36 mem.db
-rw-r--r-- 1 slot1_17 slot1_17      36864 Nov  3 15:36 time.db
-rw-r--r-- 1 slot1_17 slot1_17 5871476041 Nov  3 15:36 pdhd_prod_beam__231902_43_1_20251008T212443Z_gen_g4_IonScintPDExt.root_227040_85_1_20251010T170521Z_PDInt.root_242591_1_1_20251103T200229Z_detsim_reco1.root
-rw-r--r-- 1 slot1_17 slot1_17   54438800 Nov  3 15:34 Pandora_Events.pndr
-rw-r--r-- 1 slot1_17 slot1_17          0 Nov  3 15:06 ProtoDUNE.log
-rw-r--r-- 1 slot1_17 slot1_17          0 Nov  3 15:03 debugprod.log
-rw-r--r-- 1 slot1_17 slot1_17        519 Nov  3 15:03 detsim_single_protodunehd_hist.root
-rw-r--r-- 1 slot1_17 slot1_17 5524028354 Nov  3 15:03 temp_detsim.root
-rw-r--r-- 1 slot1_17 slot1_17        121 Nov  3 14:02 all-input-dids.txt
justIN time: 2025-11-05 07:23:17 UTC       justIN version: 01.05.01