Evaluating Transferability and Validation Protocols of Convolutional Neural Networks for Acoustic Water Leak Detection Using SebaKMT Logger Recordings

Abstract

Limited labeled data from a newly deployed logger motivate training acoustic leak classifiers on recordings from other instruments. This study evaluates direct transfer, supervised target adaptation, and validation design using SebaKMT recordings. A single-scale convolutional neural network (CNN) with 25,633 parameters is compared with two three-branch networks containing 76,769 parameters and using repeated or distinct spectral resolutions.

In the retained ten-run transfer summary, multiscale TFCNN increased sensitivity from 78.20% to 86.40%, while specificity decreased from 34.90% to 16.50%. Classifier adaptation increased test ROC-AUC from 92.50% to 94.16%, but the validation-selected threshold retained 20 alarms among 40 recordings with negative labels. This test includes a subset with an unverified historical positive label; its outcomes are therefore assessed against registry labels.

Training from scratch on 180 SebaKMT recordings yielded 100% accuracy under random file partitioning in all ten runs; this estimate is likely inflated by shared recording sessions between training and test files. Across six registered site groups (comprising seven measurement subsets), TFCNN out-of-fold accuracy was 87.83 ± 3.22% (mean within-fold ROC-AUC 94.28 ± 3.87%). Logistic regression using nine handcrafted signal descriptors, without digital amplitude, achieved an accuracy similar to that of TFCNN (87.78% vs. 87.83 ± 3.22%), with higher sensitivity (98.33% vs. 92.67%) but lower specificity (82.50% vs. 85.42%). These results do not establish an accuracy advantage for the deep architecture on this corpus. Logistic regression on digital RMS alone reached 94.44% accuracy at threshold 0.50 (within-fold ROC-AUC 99.67 ± 0.58%), although sensitivity was lower (83.33%). Digital RMS was strongly predictive of registry labels, motivating further investigation of acquisition-related shortcuts rather than confirming a physical acoustic mechanism. Transparent classical baselines should therefore be reported alongside deep models in site-grouped evaluations.

Keywords: acoustic pipeline diagnostics, leak detection, convolutional neural networks, multiscale Mel spectrograms, distribution shift, false alarms, group data leakage, site-disjoint validation, ROC-AUC, shortcut learning.

---

1. Introduction

The deployment of automated acoustic leak detection in municipal and industrial water networks often begins with limited labeled data. When an operator introduces a new fleet of correlation loggers or vibro-acoustic sensors, recordings of confirmed leaks acquired with those instruments under local soil and hydraulic conditions are generally scarce. Obtaining a representative collection of leaks verified by excavation takes time, as illustrated by long-term field campaigns on operating networks [1, 4]. A practical question is therefore whether a CNN can be trained on available recordings from other acoustic instruments and then applied to, or adapted for, a new logger using a small target dataset.

The physical rationale is that turbulent discharge through a pipe-wall defect produces broadband noise associated with cavitation and jet friction, spanning frequencies from tens of hertz to several kilohertz [1, 2]. A sensor mounted on a valve or pipe wall, however, records vibrations transmitted through the pipe, surrounding soil, and instrument rather than the source in isolation. Studies of wave propagation in buried fluid-filled pipes show that attenuation and dispersion of axisymmetric waves depend on pipe diameter, wall thickness, elastic properties, and soil damping. High-frequency components consequently attenuate with propagation distance and changes in pipe geometry [2, 3]. The piezoelectric transducer, mounting arrangement, antialiasing filter, and amplifier also determine the logger's frequency response and self-noise. Related work on recognizing environmental sounds recorded with different microphones (Mesaros et al. [12]) and detecting unusual machinery sounds under changing operating conditions (Kawaguchi et al. [13]) shows that differences in instruments and background conditions can reduce neural classifier performance.

Research on vibro-acoustic and hydroacoustic pipeline diagnostics reports high performance for deep learning, ensemble methods, and transfer learning within a sensor type [1, 4, 7] and across operating regimes or from laboratory systems to pipelines [6, 8]. False alarms on operating distribution networks remain a practical obstacle for surface-mounted accelerometers and wireless noise loggers [5, 6]. Methodological studies in clinical diagnostics, ecology, and data analysis also show that randomly partitioning individual records or segments in hierarchically related datasets can inflate performance estimates and encourage shortcut learning based on subject, site, or session characteristics [9–11, 19]. Quantifying transfer between instrument collections and session overlap in correlation-logger data is therefore relevant to deployment [9, 14–16].

We examine these issues using field recordings from SebaKMT correlation loggers. The study addresses five questions:

1. How do hidden duplicates and procedurally generated records affect the initial assessment of an unstructured source collection?
2. How does replacing a single spectral resolution with a three-resolution time–frequency representation affect sensitivity and specificity when transferring a classifier to SebaKMT recordings?
3. Can supervised adaptation of the output classifier distinguish positive recordings from stationary background, and how does threshold selection on a small validation set affect the result?
4. How do accuracy and ROC-AUC differ between random file partitioning and site-disjoint evaluation when training from scratch on SebaKMT data, and how do simple feature-based models compare?
5. How do trained models respond to auxiliary field recordings with unusual spectral patterns, including a 96 Hz tone with a historical positive label and a nighttime background survey?

2. Signal Propagation Model and Classifier Architectures

2.1. Channel Transfer Function and Time–Frequency Resolution

In a linear approximation, the complex spectrum $X(f)$ recorded by a vibro-acoustic sensor at distance $x$ from a pipe defect can be expressed as

$X(f)=S_{\mathrm{leak}}(f)\cdot H_{\mathrm{pipe}}(f,x)\cdot H_{\mathrm{sensor}}(f)+N(f),$

where $S_{\mathrm{leak}}(f)$ denotes the source spectrum, $H_{\mathrm{pipe}}(f,x)$ represents propagation through the pipe and surrounding soil, $H_{\mathrm{sensor}}(f)$ represents the mounting interface, sensing element, and electronic signal chain, and $N(f)$ denotes additive background noise from flow, pumps, traffic, and electrical interference.

This model has two implications for classification:

• In large-diameter transmission mains, represented here by a 1,000 mm pipe, high-frequency components above approximately 400–500 Hz can attenuate strongly through the pipe and soil over tens of meters [2, 3]. The informative signal may consequently be concentrated at lower frequencies, predominantly below 350 Hz in the recordings considered here.
• Changing instruments or pipe sections changes $H_{\mathrm{sensor}}(f)$, $H_{\mathrm{pipe}}(f,x)$, and $N(f)$ together. When a classifier uses spectral shape after individual amplitude normalization, stationary resonances in the background may occupy a feature region similar to that of a leak.

For the short-time Fourier transform (STFT), window duration is $N/f_s$ and frequency-bin spacing is $f_s/N$. Actual resolving power also depends on the window function. Increasing window length provides finer frequency detail while reducing temporal localization.

Short windows capture rapid amplitude fluctuations and impulsive disturbances but yield coarse frequency spacing. Long windows provide finer spacing that can distinguish narrowband interference from broader resonant components, while smoothing temporal changes. Parallel branches operating at three window lengths expose the classifier to complementary representations of the same audio segment.

2.2. Mel Spectrograms and Individual Window Normalization

Audio is converted to the operating sample rate using antialiasing filtering and divided into 3.0 s segments with a 1.5 s stride, corresponding to 50% overlap. Whole WAV files are assigned to training, validation, and test partitions before segmentation. Quality control checks finite samples, minimum RMS, crest factor, and the fraction of samples with absolute amplitude at least 0.999 of full scale. Amplitude limiting below that threshold can escape this check.

For each segment $x[n]$, three power spectrograms are computed using Hann windows of length $N_k$ and hop size $H_k$, for $k\in \{0,1,2\}$:

$S_k[m,\omega ]={\left|{\sum }_{n=0}^{N_k-1}x[n+m{H}_k]\cdot w_k[n]\cdot e^{-j\frac{2\pi \omega n}{N_k}}\right|}^2,k\in \{0,1,2\}.$

Two processing profiles correspond to the logger acquisition modes:

1. Standard profile, $f_s=6554$ Hz, analysis band 50–3000 Hz:
   • $k=0$: $N_0=256$ samples (39.1 ms), $H_0=64$ samples (9.8 ms), frequency-bin spacing $\Delta f=25.6$ Hz, 308 time frames.
   • $k=1$: $N_1=512$ samples (78.1 ms), $H_1=128$ samples (19.5 ms), $\Delta f=12.8$ Hz, 154 frames.
   • $k=2$: $N_2=1024$ samples (156.2 ms), $H_2=256$ samples (39.1 ms), $\Delta f=6.4$ Hz, 77 frames.
2. Low-frequency profile, $f_s=1638$ Hz, analysis band 30–700 Hz:
   • $k=0$: $N_0=128$ samples (78.1 ms), $H_0=32$ samples (19.5 ms), $\Delta f=12.8$ Hz, 154 frames.
   • $k=1$: $N_1=256$ samples (156.3 ms), $H_1=64$ samples (39.1 ms), $\Delta f=6.4$ Hz, 77 frames.
   • $k=2$: $N_2=512$ samples (312.6 ms), $H_2=128$ samples (78.1 ms), $\Delta f=3.2$ Hz, 39 frames.

Each spectrogram is projected onto a bank of $M=64$ triangular Mel filters $H_b(\omega )$:

${\tilde{M}}_k[m,b]={\sum }_{\omega }{S}_k[m,\omega ]\cdot H_b(\omega ),b\in \{1,\dots ,64\}.$

Power is converted to decibels relative to the maximum of each spectrogram, limited to an 80 dB range, and linearly mapped to $[0,1]$:

$M_k^{\mathrm{dB}}[m,b]=\max \left(-80,10{\log }_{10}\frac{{\tilde{M}}_k[m,b]+\epsilon }{{\max }_{m^{\prime },b^{\prime }}{\tilde{M}}_k[m^{\prime },b^{\prime }]}\right),M_k^{\mathrm{norm}}[m,b]=\frac{M_k^{\mathrm{dB}}[m,b]+80}{80}\in [0,1].$

The time axes of $M_0^{\mathrm{norm}}$ and $M_2^{\mathrm{norm}}$ are resampled by bilinear interpolation to $T=154$ frames, producing a $3\times 64\times 154$ input tensor.

Fig. 1. Examples of the actual network input for registry-positive subset L1 (a) and registry-negative subset N3 (b). The middle resolution is shown after the shared bandpass filter and extraction of a 3 s segment: 6554 Hz, FFT length 512, hop size 128, and 64 Mel bands. Color represents log power relative to each spectrogram's maximum, mapped to [0, 1].

Normalization to each spectrogram's maximum removes a global amplitude scaling factor, including a linear gain change, but discards information about absolute digital RMS. In this SebaKMT corpus, positive subsets ($L_1,L_4,L_5$) have mean RMS levels between −15.9 and −1.4 dBFS, whereas negative subsets ($N_1$–$N_4$) have lower mean levels, between −35.0 and −27.6 dBFS. After normalization, a quiet narrowband background reaches the same maximum color value as an intense positive recording (Figure 1b).

2.3. Neural Architectures and Conventional Baselines

Three architectures are compared to distinguish the effect of multiple spectral resolutions from that of increased parameter count (Figure 2):

1. Single-scale CNN, 25,633 parameters. The input is the middle-resolution spectrogram ($1\times 64\times 154$). One convolutional branch contains 23,520 parameters and three blocks with 16, 32, and 64 channels. Each block uses a $3\times 3$ convolution, batch normalization, ReLU activation, and $2\times 2$ pooling. Global average pooling produces a 64-element feature vector. A two-layer classifier comprises $\mathrm{Linear}(64\to 32)$, ReLU, dropout with $p=0.3$, and $\mathrm{Linear}(32\to 1)$. The classifier contains 2,113 parameters, giving 25,633 overall.
2. Three-resolution TFCNN, 76,769 parameters. Three independent branches, each containing 23,520 parameters, process resolutions $k=0,1,2$. Their three 64-element vectors are concatenated into a 192-element representation. The classifier comprises $\mathrm{Linear}(192\to 32)$, ReLU, dropout with $p=0.3$, and $\mathrm{Linear}(32\to 1)$, totaling 6,209 parameters. The network therefore contains $3\times 23,520+6,209=76,769$ parameters.
3. Capacity control, 76,769 parameters. The three-branch topology and parameter count match TFCNN, but all branches receive the same middle-resolution input, $[M_1^{\mathrm{norm}},M_1^{\mathrm{norm}},M_1^{\mathrm{norm}}]$.

Fig. 2. Single-scale CNN (a, 25,633 parameters) and three-resolution TFCNN (b, 76,769 parameters). The capacity control uses the topology in panel b but supplies the middle-resolution spectrogram to every branch. A file score is the arithmetic mean of sigmoid outputs across its 3 s windows.

Two conventional baselines were also evaluated on the primary SebaKMT corpus: logistic regression with $L_2$ regularization and standardized features, and a random forest with 100 trees. Ten signal features were computed per WAV file: ${\log }_{10}\mathrm{RMS}$, crest factor, zero-crossing rate, Wiener spectral flatness over 50–3000 Hz, spectral centroid, spectral bandwidth, and four relative energy fractions over 50–150, 150–400, 400–1000, and 1000–3000 Hz. Spectral features were derived using Welch's method with 1,024-sample windows.

The baselines used three feature configurations to examine the potential role of digital level as a shortcut:

1. Amplitude only, comprising ${\log }_{10}\mathrm{RMS}$.
2. Nine shape descriptors excluding ${\log }_{10}\mathrm{RMS}$. This is a separate control without an explicit digital-level feature; it is not equivalent to changing CNN normalization.
3. All ten features, including ${\log }_{10}\mathrm{RMS}$.

3. Data and Evaluation Protocols

3.1. Audit of the Source Collection

The source collection from other instruments initially contained several problems common to unstructured operational archives:

• Of 203 training and validation WAV files, only 141 were byte-unique. File and decoded-sample SHA-256 checks identified 62 exact duplicates stored under different names or paths.
• Before deduplication, 31 of 61 validation files had identical copies in training. They accounted for approximately 54% of validation duration, or 3,441 of 6,345 s.
• Conflicting labels, dependence between sample rate and class, and approximately 100 procedurally generated records with nonphysical spectral templates were also identified in the negative folder.

After deduplication, exclusion of generated records, quarantine of disputed labels, and stratified partitioning, the curated collection contained 136 unique recordings. Standard-profile quality control retained 133 files, yielding 6,380 three-second windows: 88 training files (3,907 windows), 24 validation files (1,419 windows), and 21 test files (1,054 windows; 14 positive and seven negative files).

3.2. SebaKMT Field Recordings and Group Structure

The target collection comprises SebaKMT Correlux / K-2 field recordings attributed through correlator project files and an operational registry. Standard-profile files are sampled at 6554 Hz and last approximately 3.12 s. Each provides one complete 3 s window; the remaining approximately 0.125 s is unused. Low-frequency files are sampled at 1638 Hz and last 12.50 s, providing seven overlapping 3 s windows per file.

The primary target corpus contains 180 standard-profile recordings from six site groups, with 60 positive and 120 negative registry labels. Additional analyses consider 30 low-frequency recordings of one leak event and 39 auxiliary recordings with historical labels.

In the primary corpus, subsets $L_4$ and $N_4$ were recorded on supply pipes within the same large network junction and were assigned to one site group. The auxiliary collection contains subsets $T_1$ and $T_2$, totaling 39 WAV files.

Short recordings from one session share acquisition conditions. The primary corpus contains seven registry subsets across six site groups and three registered leak events. The WAV count is not the number of independent field trials. Separation was established using registered identifiers, without an independent geospatial survey of all physical connections.

3.3. Experimental and Statistical Protocols

Three sequential protocols were used:

1. Protocol A, direct transfer between collections. The single-scale CNN, capacity control, and TFCNN were developed on the curated source corpus of 133 files with ten random initializations, seeds 42–51. Thresholds were fixed using source validation data. Without target fine-tuning, the models were evaluated on 150 SebaKMT files: 50 positive recordings, comprising 20 from $L_1$ and 30 from $L_2$, and 100 negative recordings from $N_1,N_2,N_3$. The retained analysis compared architectures using paired two-sided Student's $t$-tests across seeds with Holm–Bonferroni correction.
2. Protocol B, threshold selection and classifier adaptation. Original pretrained weights and an adapted output classifier were compared on fixed SebaKMT partitions.
   • In the standard profile, training comprised 79 accepted files: $L_4$ (20), $N_1$ (20), $N_4$ (20), and $T_2$ (19). Validation comprised $L_5$ (20) and $N_2$ (40). Testing comprised $L_1$ (20), $T_1$ (20), and $N_3$ (40). The classifier was fine-tuned for 15 epochs while convolutional branches were frozen. The historical positive label of $T_1$ was not independently verified, so test metrics describe agreement with registry labels. The threshold was selected subject to ${\mathrm{Recall}}_{\mathrm{val}}\ge 95\%$.
   • The low-frequency profile used 30 training, 50 validation, and 50 test files. The 30 recordings of event $L_2$ were partitioned by sensor: ten from sensor 1 for training, ten from sensor 2 for validation, and ten from sensor 3 for testing. All three sensors recorded the same event on the same main. Relative to generalization to a new leak event, this design therefore contains within-event data leakage.
3. Protocol C, training from scratch on SebaKMT data with random-file and site-group partitions. All TFCNN parameters and the conventional baselines were trained using the 180-file primary corpus. Both partitioning strategies used 60 training, 60 validation, and 60 test files, with 20 positive and 40 negative files per partition.
   • Site-disjoint evaluation used three rotating blocks and ten seeds, giving 30 TFCNN models. Block A contained $L_4$ (20), $N_4$ (20), and $N_1$ (20), keeping the two subsets from the large junction together. Block B contained $L_1$ (20) and $N_2$ (40). Block C contained $L_5$ (20) and $N_3$ (40). Each block served in turn as training, validation, and testing. For each seed, predictions from the three external test blocks were combined into an out-of-fold assessment of all 180 files.
   • A random-file control used the same corpus and class counts with a 60/60/60 partition and ten seeds. Training settings were matched: AdamW, dropout 0.2 rather than the original architecture's 0.3, at most 60 epochs, early stopping after ten epochs without improvement in class-balanced file-level validation cross-entropy, and a threshold satisfying ${\mathrm{Recall}}_{\mathrm{val}}\ge 0.95$.

The standard deviation across ten seeds characterizes optimization variability at fixed partitions. Accuracy across the seven registry subsets is reported descriptively. Two subsets share a site, and some subsets combine multiple recorded series; they are not seven independent field trials. No confidence interval for performance at future sites is inferred from these subset summaries.

4. Results

4.1. Direct Transfer Between Collections Under Protocol A

Table 1 and Figure 3 summarize direct transfer to 150 SebaKMT files: 50 positive recordings, comprising 20 from $L_1$ and 30 from low-frequency main $L_2$, and 100 negative recordings from $N_1,N_2,N_3$. An always-negative classifier provides a reference accuracy of 66.67% because of the 1:2 class ratio, with balanced accuracy (BA) of 50.00%.

Table 1. Direct transfer from the source collection to 150 SebaKMT WAV files without target adaptation. Values are mean ± SD across ten seeds, 42–51, unless only a mean is shown.

Fig. 3. Mean confusion matrices across ten seeds for direct transfer to 150 SebaKMT files with 50 positive and 100 negative labels. Panels show the single-scale CNN (a, 25,633 parameters), repeated-scale capacity control (b, 76,769 parameters), and multiscale TFCNN (c, 76,769 parameters).

The retained results support two observations:

1. Sensitivity with multiple spectral resolutions. Mean sensitivity across all 50 positive recordings increased from 78.20 ± 7.21% for the single-scale model to 86.40 ± 8.21% for TFCNN. The retained paired test gave $t=2.051$, $p=0.0704$, which does not meet the 5% significance criterion. Sensitivity was also higher than for the equal-size capacity control, 73.80 ± 6.43%, with unadjusted $t=3.324$, $p=0.0089$. On the 1,000 mm low-frequency main $L_2$, TFCNN detected a mean of 23.2 ± 4.1 of 30 positive files (77.3%), compared with 19.1 ± 3.6 (63.7%) for the single-scale model ($t=2.772$, $p=0.0217$) and 18.0 ± 5.0 (60.0%) for the capacity control ($t=2.376$, $p=0.0415$). TFCNN used three resolutions, including a 312.6 ms window with 3.2 Hz frequency-bin spacing. For a family of six paired comparisons, however, Holm–Bonferroni correction raises the smallest unadjusted $p=0.0089$ to $p_{\mathrm{adj}}=0.0534$. This exceeds $\alpha =0.05$, so the observation is exploratory.
2. Low specificity and substantial seed variation. The single-scale CNN produced 65.1 ± 38.9 false positives per 100 negative files, compared with 83.5 ± 28.1 for TFCNN. Specificity was 34.90% and 16.50%, respectively; the retained comparison was not statistically significant ($t\approx -1.21$, $p\approx 0.24$). Balanced accuracy was 56.55% for the single-scale model and 51.45% for TFCNN. At these operating points, direct transfer did not provide performance suitable for practical deployment.

4.2. Threshold Selection and Supervised Adaptation Under Protocol B

Table 2 compares the pretrained TFCNN, threshold adjustment, and output-classifier adaptation on fixed target partitions. The standard-profile test contains 40 positive-labeled files from $L_1$ and $T_1$ and 40 negative-labeled files from $N_3$. The low-frequency test contains ten files from a held-out sensor recording event $L_2$ and 40 resampled negative files from $N_3$. Validation and test ROC-AUC are reported alongside threshold-dependent outcomes.

Table 2. Pretrained TFCNN, target-validation threshold selection, and supervised classifier adaptation on SebaKMT recordings. Adapted results summarize ten seeds, except for the explicitly post hoc seed-42 illustration.

Fig. 4. File-score distributions in the standard profile before and after output-classifier adaptation. The upper row shows validation (60 WAV files); the lower row shows testing (40 positive-labeled and 40 negative-labeled files). Columns show the pretrained model and the adapted model for seed 42. Blue and orange indicate negative and positive registry labels. Dashed lines mark thresholds selected on validation subject to sensitivity of at least 95%.

Two aspects of performance should be distinguished:

1. Test-score ranking. Before adaptation, 20 files from one of the two recorded series in negative subset $N_3$ had scores between 0.50 and 1.00, overlapping the positive class; test ROC-AUC was 92.50%. After classifier adaptation, test ROC-AUC increased to 94.16 ± 0.03%. In seed 42, most of these background scores shifted to approximately 0.40–0.76. Three negative files remained above 0.78, with scores of 0.897, 0.912, and 0.937, while 37 of 40 positive-labeled files lay between 0.79 and 0.99. A threshold of 0.78 would produce 37/40 true positives and 3/40 false positives, corresponding to 92.50% sensitivity, specificity, and balanced accuracy. This threshold was examined after inspecting the test data and is not an independently validated operating rule.
2. Sensitivity of threshold selection to low-scoring validation records. Every adapted run produced 20 false positives among 40 negative test files at its validation-selected threshold. Two positive-labeled validation files from $L_5$ (zelenaya, 040007AC01.wav and 040007AC04.wav) received scores of 0.0242 and 0.0935 in seed 42. They do not belong to disputed subset $T_1$, and a low score alone does not demonstrate a labeling error. With 20 positive validation files, the requirement ${\mathrm{Recall}}_{\mathrm{val}}\ge 95\%$ permits only one missed file. The selected threshold must therefore include at least the second-lowest positive score. Across seeds, the threshold was $0.0667\pm 0.0015$. All 20 files from the higher-scoring negative series in $N_3$, with test scores between 0.126 and 0.937, exceeded this threshold.

4.3. Training from Scratch and Comparison with Conventional Baselines Under Protocol C

To assess performance without the source collection, all TFCNN parameters were trained from scratch using SebaKMT recordings. Preliminary random-file experiments had produced test accuracies of 99.5–100%.

The curated 180-file corpus, containing 60 positive and 120 negative labels, was then used to compare random-file partitioning with three-block site-disjoint evaluation. Ten seeds per grouped split produced 30 TFCNN models. Conventional baselines used the same grouped splits and three feature configurations: digital RMS alone, nine handcrafted signal descriptors without RMS, and all ten features. Table 3 and Figure 5 summarize the comparison.

Table 3. TFCNN and conventional baselines on the 180-file SebaKMT corpus. TFCNN includes random-file and site-group protocols; conventional baselines use the grouped splits. Seed-dependent results summarize seeds 42–51.

For CNN and random forest, ± denotes sample SD across ten seeds. For within-fold ROC-AUC of deterministic logistic regression, ± denotes SD across the three folds. These quantities describe different sources of variation and are not directly comparable. Pooled ROC-AUC combines scores from separately trained models and is provided descriptively.

Fig. 5. Mean confusion matrices across ten seeds for TFCNN trained from scratch on SebaKMT recordings. Random-file partitioning uses 60 test files (a); site-group partitioning combines external predictions for 180 files (b). Percentages are normalized within each registry class.

Five observations follow:

1. Evaluation protocol. The random-file control achieved 100% accuracy. Grouped evaluation yielded accuracy of 87.83 ± 3.22%, sensitivity of 92.67 ± 5.10%, and specificity of 85.42 ± 5.22%. Mean errors were 17.50 ± 6.26 false positives among 120 negative files and 4.40 ± 3.06 false negatives among 60 positive files. The 12.17 percentage-point accuracy difference describes these protocols with different test compositions; it does not isolate a causal effect of session overlap. Conventional models were evaluated only on the grouped splits in this analysis.
2. Baseline model comparison. Logistic regression using nine handcrafted signal descriptors achieved an accuracy similar to that of TFCNN (87.78% vs. 87.83 ± 3.22%), with higher sensitivity (98.33% vs. 92.67%) but lower specificity (82.50% vs. 85.42%). These results do not establish an accuracy advantage for the deep architecture on this corpus. Logistic regression on digital RMS alone reached 94.44% accuracy at threshold 0.50 (within-fold ROC-AUC 99.67 ± 0.58%), though with lower sensitivity (83.33% vs. 92.67%). Digital RMS was strongly predictive of registry labels, motivating further investigation of acquisition-related shortcuts rather than demonstrating a physical acoustic mechanism.
3. Descriptive variation across subsets. Unweighted mean accuracy across the seven registry subsets was 87.07%, with an SD of 17.21 percentage points between subset means. This describes heterogeneity in the current corpus and is not a confidence interval for future sites.
4. Digital level and possible amplitude limiting. In $L_1$, a mean of 51.89% of samples had an absolute amplitude of at least 0.99 full scale, and the mean crest factor was 1.18. Only 1.99% of samples had an absolute raw code of exactly 32688. These are distinct quantities. The waveform is consistent with amplitude limiting, but its origin in the analog-to-digital converter, analog signal chain, digital processing, or export cannot be established. The quality-control threshold of 0.999 does not detect these near-full-scale samples. Predictive digital RMS may reflect signal intensity, acquisition conditions, or both; gain settings were not recovered from the WAV files.
5. Threshold selection for TFCNN. With a fixed threshold of 0.50, predictions from the three grouped models yielded 63.25% specificity and 72.78% accuracy. Selecting a threshold on each validation block subject to sensitivity of at least 95% adjusted the operating point and yielded 87.83% overall accuracy.

Table 4. Acoustic characteristics and out-of-fold accuracy across seven registry subsets.

Logistic regression uses all ten features and thresholds selected only on validation. Correct predictions summed across subsets are 170/180 (94.44%), with ten false positives and no false negatives, matching Table 3. At a fixed threshold of 0.50, accuracy on $N_4$ is 90% and the total is 168/180 (93.33%). SDs of RMS, crest factor, and the high-amplitude sample fraction are calculated across files. The high-amplitude sample mean is expressed as a percentage and its SD in percentage points.

Fig. 6. Out-of-fold errors across ten random initializations, seeds 42–51, under grouped evaluation of 180 WAV files. Panels show false positives among 120 negative files (a, mean 17.5) and false negatives among 60 positive files (b, mean 4.4).

Fig. 7. TFCNN accuracy across seven registry subsets under grouped evaluation. Bars indicate means across ten seeds; whiskers span the minimum and maximum. Panels show subsets with positive (a) and negative (b) registry labels. These subsets are not seven independent sites.

The subset analysis identifies three additional patterns:

1. Fewer alarms on $N_3$ after training from scratch. Under classifier adaptation in Protocol B, $N_3$ produced 20 alarms among 40 negative-labeled files (50.0%). In grouped split 3, with $N_3$ excluded from both training and validation, training from scratch produced 4.80 ± 3.08 false positives among these 40 files (12.00%), with fold ROC-AUC of 94.67 ± 1.75%.
2. Heterogeneity within $N_4$. TFCNN produced 9.90 ± 0.32 alarms among 20 files, ranging from nine to ten. Alarms were concentrated on sensor A; sensor B produced none. The source of this asymmetry remains unresolved. The current ten-feature logistic regression produced two alarms on $N_4$ at threshold 0.50 and none at its validation-selected threshold. It therefore did not reproduce the same ten alarms as TFCNN.
3. Early-stopping instability with few events. In grouped split 2, seeds 42–50 produced zero to four false positives among 40 negative files in $N_2$, with test ROC-AUC of 97.9–100.0%. For seed 51, validation on block A selected the first-epoch checkpoint; test ROC-AUC was 65.5%, with 21 false positives among 40 files. This contributes to the peak of 33 overall false positives in Figure 6a. Each training partition contains only one positive event, represented by 20 WAV files, making epoch and threshold selection sensitive to initialization and the single validation event.

4.4. Auxiliary Field Recordings T1 and T2

After models and thresholds had been fixed, an auxiliary analysis was performed on 39 recordings excluded from training these 40 models. The files had been used in earlier development, and their physical grouping and labels were not independently verified. This is therefore neither a new blind evaluation nor an established independent out-of-distribution test.

Fig. 8. Alarm counts on auxiliary subsets T1 (a, 20 WAV files) and T2 (b, 19 WAV files). Bars and error bars indicate means and sample SDs across ten seeds for each split. Historical labels are not treated as independently verified ground truth.

1. Subset T1. These 20 historically positive-labeled recordings contain a prominent tone near 96 Hz. After the subset was excluded from training, 39 of 40 models produced no alarms and one produced eight. This change relative to earlier runs establishes neither the physical source of the signal nor memorization of that particular tone. Without label verification, it cannot be interpreted as a confirmed leak miss rate.
2. Subset T2. For the 19 historically negative-labeled recordings, mean alarm counts were 5.50 ± 1.78, 6.20 ± 5.27, and 0.20 ± 0.42 for the three grouped splits, and 0.50 ± 1.58 for the random control. These are alarm counts relative to historical labels, not independently verified false-alarm rates.

5. Discussion

5.1. Distribution Shift and Digital Signal Level

Direct transfer changes the instrument, pipe section, background, and acquisition conditions together. Peak normalization removes a global digital amplitude factor but does not make different measurement systems equivalent. Its contribution to network errors remains a hypothesis because a paired normalization ablation was not performed in this comparison.

Conventional models show that digital RMS is associated with registry class in this corpus. Its predictive performance does not establish a universal physical relationship or identify the cause of the level difference. Digital dBFS is not calibrated sound pressure. Peak normalization also preserves waveform distortions introduced by amplitude limiting, which may remain potential shortcut features for a CNN.

Published neural and ensemble approaches [4–8] use different datasets and acquisition conditions. Their numerical results cannot be directly ranked against this study without examining site composition, evaluation protocols, and label verification. Research on domain shift [12, 13] and normalization [17, 18] provides directions for further investigation but does not establish the cause of the errors observed here.

5.2. Score Ranking and Operating Thresholds

ROC-AUC measures the ranking of model scores relative to labels. In the standard profile, adaptation improved test ROC-AUC to 94.16% but did not reduce alarms on negative-labeled recordings at the validation-selected threshold. Two low scores in validation subset $L_5$ affected the threshold when at least 19 of 20 positive validation files had to be detected. Their low scores do not demonstrate label errors.

A post hoc threshold of 0.78 reduces errors on this test, but it was not selected independently of test labels. Alternative rules, including maximizing Youden's $J$, can be compared using validation data; they do not guarantee stability at a new site. Selecting a threshold is distinct from calibrating model output as a probability. The low-frequency result concerns a sensor recording the same event and does not establish transfer to a new leak.

5.3. Validation Design and Causal Interpretation

Work on grouped validation and data leakage [9–11, 14–16] supports matching data partitioning to the intended application. Random-file partitioning retains familiar acquisition conditions in training and testing. In our comparison, its performance exceeded that obtained when registered site groups were excluded.

This is consistent with the use of session-related features but does not identify the features learned by the network. The random control uses 60 test files, whereas the pooled grouped assessment uses 180; the 12.17 percentage-point difference is not a paired causal estimate. Similarly, comparing classifier adaptation with training from scratch changes both data composition and training settings and does not isolate the benefit or harm of pretraining.

6. Limitations and Future Work

1. Event count. The primary corpus contains three registered leak events. Each split places one event in training, one in validation, and one in testing. Epoch and threshold selection depend on one validation event. Seven registry subsets belong to six site groups; seed variation does not quantify uncertainty across future sites.
2. Labels and prior exposure. Labels were inherited from the registry and were not independently verified in the field during this study. The adaptation test includes disputed subset $T_1$. The 39 auxiliary recordings were used in earlier development. Incomplete file-level records of early transfer runs prevent independent reproduction of all paired tests.
3. Digital signal level. High digital levels and signs consistent with amplitude limiting motivate investigation of acquisition-related features but do not establish their hardware origin. Gain settings and export parameters are unknown. Waveform changes may survive peak normalization and require controlled investigation.
4. Further controls. Future work should prespecify a rule for identifying windows without near-full-scale samples and report the fraction of data retained; assess sensitivity to excluding $L_1$; and examine the effects of artificial amplitude limiting on both classes. These controls have not been performed and do not replace new field recordings. Records should not be excluded based on their test predictions.
5. Independent evaluation. Performance depends on dataset composition and the selected threshold. Assessing new sites requires independently collected recordings with verified labels and a prespecified protocol. Model weights, processing, thresholds, and metrics should be fixed before inspecting new-site outcomes. Adaptation data must be separated from the final test by event and site. External evaluation should include leaks and diverse background conditions, with quality-control rejections reported separately.

---

7. Conclusions

1. Direct transfer of the three-resolution network increased sensitivity relative to the single-scale model but yielded low specificity and did not establish reliable transfer to new instruments and sites.
2. Output-classifier adaptation improved ranking on the standard-profile test but retained 20 alarms among 40 negative-labeled files at the validation-selected threshold. These metrics describe agreement with registry labels, including disputed subset $T_1$.
3. Training from scratch achieved 100% accuracy under random-file partitioning and 87.83 ± 3.22% under grouped evaluation. This contrast demonstrates dependence on evaluation design without establishing a specific memorization mechanism.
4. Responses to 39 auxiliary recordings depended on training context. Without verified labels, these results describe alarms rather than confirmed field-diagnostic errors.
5. Logistic regression using nine handcrafted signal descriptors achieved an accuracy similar to that of TFCNN (87.78% vs. 87.83 ± 3.22%), with higher sensitivity (98.33% vs. 92.67 ± 5.10%) but lower specificity (82.50% vs. 85.42 ± 5.22%). These results do not establish an accuracy advantage for the deep architecture on this corpus. Logistic regression on digital RMS alone reached 94.44% accuracy at threshold 0.50 and 99.67 ± 0.58% within-fold ROC-AUC, but with lower sensitivity (83.33%), which is a notable operational caveat for leak detection. Digital RMS was strongly predictive of registry labels, motivating further investigation of acquisition-related shortcuts (such as gain settings, converter limits, or export processing) combined with recording anomalies, rather than confirming a physical acoustic mechanism. Site-grouped benchmarks on small datasets should therefore report transparent feature baselines alongside deep models.

---

Acknowledgments and Declaration of Generative AI Usage

During the preparation of this work, the authors used Generative AI and Large Language Model (LLM) tools solely as an assistive technology for cross-lingual drafting, technical translation from the authors' primary working language, grammatical refinement, and manuscript formatting. All scientific hypotheses, physical and mathematical problem formulations, experimental software implementations, empirical data processing, and analytical conclusions were independently developed, executed, and verified by the authors. The authors retain full personal and academic responsibility for the integrity and content of this publication.

---

References

1. Bykerk, L., Valls Miro, J. (2022). Vibro-Acoustic Distributed Sensing for Large-Scale Data-Driven Leak Detection on Urban Distribution Mains. Sensors, 22(18), 6897. DOI: 10.3390/s22186897.
2. Muggleton, J. M., Brennan, M. J., Pinnington, R. J. (2002). Wavenumber prediction of waves in buried pipes for water leak detection. Journal of Sound and Vibration, 249(5), 939–954. DOI: 10.1006/jsvi.2001.3881.
3. Muggleton, J. M., Brennan, M. J., Linford, P. W. (2004). Axisymmetric wave propagation in fluid-filled pipes: wavenumber measurements in in vacuo and buried pipes. Journal of Sound and Vibration, 270(1–2), 171–190. DOI: 10.1016/S0022-460X(03)00489-9.
4. Gunatilake, A., Valls Miro, J. (2024). Multimodel Neural Network for Live Classification of Water Pipe Leaks From Vibro-Acoustic Signals. IEEE Sensors Journal, 24(9), 14825–14832. DOI: 10.1109/JSEN.2024.3379476.
5. Boadu, V., El-Zahab, S., Zayed, T. (2024). Enhancing Automated Acoustic Leak Detection in a Water Distribution Network Using Ensemble Machine Learning. Journal of Water Resources Planning and Management (ASCE), 150(11). DOI: 10.1061/JWRMD5.WRENG-6578.
6. Bakhtawar, B., Fares, A., Zayed, T. (2025). AIoT-Driven Leak Detection in Real Water Networks Using Hydrophones. Water Resources Management, 39(6), 2551–2566. DOI: 10.1007/s11269-024-04077-3.
7. Kwon, S., Jeon, S., Park, T.-J., Bae, J.-H. (2024). Automatic Weight Redistribution Ensemble Model Based on Transfer Learning to Use in Leak Detection for the Power Industry. Sensors, 24(15), 4999. DOI: 10.3390/s24154999.
8. Cheng, J., Jiang, Z., Wu, H., Zhang, X. (2025). Water Pipeline Leak Detection Method Based on Transfer Learning. Water, 17(3), 368. DOI: 10.3390/w17030368.
9. Saeb, S., Lonini, L., Jayaraman, A., Mohr, D. C., Kording, K. P. (2017). The need to approximate the use-case in clinical machine learning. GigaScience, 6(5), gix019. DOI: 10.1093/gigascience/gix019.
10. Little, M. A., Varoquaux, G., Saeb, S., et al. (2017). Using and understanding cross-validation strategies. Perspectives on Saeb et al. GigaScience, 6(5), gix020. DOI: 10.1093/gigascience/gix020.
11. Roberts, D. R., Bahn, V., Ciuti, S., et al. (2017). Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure. Ecography, 40(8), 913–929. DOI: 10.1111/ecog.02881.
12. Mesaros, A., Heittola, T., Virtanen, T. (2018). A multi-device dataset for urban acoustic scene classification. Proceedings of the Detection and Classification of Acoustic Scenes and Events 2018 Workshop (DCASE2018), 9–13. arXiv:1807.09840.
13. Kawaguchi, Y., Imoto, K., Koizumi, Y., et al. (2021). Description and Discussion on DCASE 2021 Challenge Task 2: Unsupervised Anomalous Sound Detection for Machine Condition Monitoring under Domain Shifted Conditions. Proceedings of the DCASE 2021 Workshop, 186–190. arXiv:2106.04492.
14. Tougui, I., Jilbab, A., El Mhamdi, J. (2021). Impact of the Choice of Cross-Validation Techniques on the Results of Machine Learning-Based Diagnostic Applications. Healthcare Informatics Research, 27(3), 189–199. DOI: 10.4258/hir.2021.27.3.189.
15. Kaufman, S., Rosset, S., Perlich, C., Stitelman, O. (2012). Leakage in data mining: Formulation, detection, and avoidance. ACM Transactions on Knowledge Discovery from Data (TKDD), 6(4), 15. DOI: 10.1145/2382577.2382579.
16. Kapoor, S., Narayanan, A. (2023). Leakage and the reproducibility crisis in machine-learning-based science. Patterns, 4(9), 100804. DOI: 10.1016/j.patter.2023.100804.
17. Lostanlen, V., Palmer, K., Knight, E., et al. (2019). Per-Channel Energy Normalization: Why and How. IEEE Signal Processing Letters, 26(1), 39–43. DOI: 10.1109/LSP.2018.2878620.
18. Li, Y., Wang, N., Shi, J., Hou, X., Liu, J. (2018). Adaptive Batch Normalization for practical domain adaptation. Pattern Recognition, 80, 109–117. DOI: 10.1016/j.patcog.2018.03.005.
19. Geirhos, R., Jacobsen, J.-H., Michaelis, C., et al. (2020). Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11), 665–673. DOI: 10.1038/s42256-020-00257-z.

---

Citation

This research paper is permanently archived as a preprint on Zenodo:

DOI: 10.5281/zenodo.23446900

Ivanaiskii, A., Ivanaiskii, E., & Shipilov, S. (2026). Evaluating Transferability and Validation Protocols of Convolutional Neural Networks for Acoustic Water Leak Detection Using SebaKMT Logger Recordings [Preprint]. Zenodo. https://doi.org/10.5281/zenodo.23446900