Found 15510 results, showing the newest relevant preprints. Sort by relevancy only.Update me on new preprints

Predicting trends in the quality of state-of-the-art neural **networks**
without access to training or testing data

In many applications, one works with deep neural

**network**(DNN) models trained by someone else. Expand abstract. In many applications, one works with deep neural

**network**(DNN) models trained by someone else. For such pretrained models, one typically does not have access to training/test data. Moreover, one does not know many details about the model, such as the specifics of the training data, the loss function, the hyperparameter values, etc. Given one or many pretrained models, can one say anything about the expected performance or quality of the models? Here, we present and evaluate empirical quality metrics for pretrained DNN models at scale. Using the open-source WeightWatcher tool, we analyze hundreds of publicly-available pretrained models, including older and current state-of-the-art models in CV and NLP. We examine norm-based capacity control metrics as well as newer Power Law (PL) based metrics (including fitted PL exponents and a Weighted Alpha metric), from the recently-developed Theory of Heavy-Tailed Self Regularization. Norm-based metrics correlate well with reported test accuracies for well-trained models across nearly all CV architecture series. On the other hand, norm-based metrics can not distinguish "good-versus-bad" models---which, arguably is the point of needing quality metrics. Indeed, they may give spurious results. PL-based metrics do much better---quantitatively better at discriminating series of "good-better-best" models, and qualitatively better at discriminating "good-versus-bad" models. PL-based metrics can also be used to characterize fine-scale properties of models, and we introduce the layer-wise Correlation Flow as new quality assessment. We show how poorly-trained (and/or poorly fine-tuned) models may exhibit both Scale Collapse and unusually large PL exponents, in particular for recent NLP models. Our techniques can be used to identify when a pretrained DNN has problems that can not be detected simply by examining training/test accuracies.8 days ago

6/10 relevant

arXiv

6/10 relevant

arXiv

ArcText: An Unified Text Approach to Describing Convolutional Neural
**Network** Architectures

Numerous Convolutional Neural

**Network**(CNN) models have demonstrated their promising performance mostly in computer vision. Expand abstract. Numerous Convolutional Neural

**Network**(CNN) models have demonstrated their promising performance mostly in computer vision. The superiority of CNNs mainly relies on their complex architectures that are often manually designed with extensive human expertise. Data mining on CNN architectures can discover useful patterns and fundamental sub-comments from existing CNN architectures, providing common researchers with strong prior knowledge to design CNN architectures when they have no expertise in CNNs. There have been various state-of-the-art data mining algorithms at hand, while there is rare work that has been used for this aspect. The main reason behind this is the barrier between CNN architectures and data mining algorithms. Specifically, the current CNN architecture descriptions cannot be exactly vectorized to the input to data mining algorithms. In this paper, we propose a unified approach, named ArcTxt, to describing CNN architectures based on text. Particularly, three different units of ArcText and an order method have been elaborately designed, to uniquely describe the same architecture including the sufficient information. Also, the resulted description can also be exactly converted back to the corresponding CNN architecture. ArcText bridge the gap between CNN and data mining researchers, and has the potentiality to be utilized to wider scenarios.8 days ago

5/10 relevant

arXiv

5/10 relevant

arXiv

Proteogenomic heterogeneity of localized human prostate cancer progression

Overall, this study revealed molecular

**networks**with remarkably convergent alterations across tumor sites and patients, but it also exposed a diversity of**network**effects: we could not identify a single sub-network that was perturbed in all high-grade tumor regions. Expand abstract. Tumor-specific genomic aberrations are routinely determined by high throughput genomic measurements. However, it is unclear how complex genome alterations affect molecular

**networks**through changing protein levels, and consequently biochemical states of tumor tissues. Here, we investigated how tumor heterogeneity evolves during prostate cancer progression. In this study, we performed proteogenomic analyses of 105 prostate samples, consisting of both benign prostatic hyperplasia regions and malignant tumors, from 39 prostate cancer (PCa) patients. Exome sequencing, copy number analysis, RNA sequencing and quantitative proteomic data were integrated using a**network**-based approach and related to clinical and histopathological features. In general, the number and magnitude of alterations (DNA, RNA and protein) correlated with histopathological tumor grades. Although common sets of proteins were affected in high-grade tumors, the extent to which these proteins changed their concentrations varied considerably across tumors. Our multi-layer**network**integration identified a sub-**network**consisting of nine genes whose activity positively correlated with increasingly aggressive tumor phenotypes. Importantly, although the effects on individual gene members were barely detectable, together the perturbation of this sub-**network**was predictive for recurrence-free survival time. The multi-omics profiling of multiple tumor sites from the same patients revealed cases of likely shared clonal origins as well as the occasional co-existence of multiple clonally independent tumors in the same prostate. Overall, this study revealed molecular**networks**with remarkably convergent alterations across tumor sites and patients, but it also exposed a diversity of**network**effects: we could not identify a single sub-**network**that was perturbed in all high-grade tumor regions.9 days ago

6/10 relevant

bioRxiv

6/10 relevant

bioRxiv

Autonomous Unknown-Application Filtering and Labeling for DL-based Traffic Classifier Update

Specifically, Deep Learning (DL) has attracted much attention from the researchers due to its effectiveness even in encrypted

**network**traffic without compromising neither user privacy nor network security. Expand abstract.**Network**traffic classification has been widely studied to fundamentally advance

**network**measurement and management. Machine Learning is one of the effective approaches for

**network**traffic classification. Specifically, Deep Learning (DL) has attracted much attention from the researchers due to its effectiveness even in encrypted

**network**traffic without compromising neither user privacy nor

**network**security. However, most of the existing models are created from closed-world datasets, thus they can only classify those existing classes previously sampled and labeled. In this case, unknown classes cannot be correctly classified. To tackle this issue, an autonomous learning framework is proposed to effectively update DL-based traffic classification models during active operations. The core of the proposed framework consists of a DL-based classifier, a self-learned discriminator, and an autonomous self-labeling model. The discriminator and self-labeling process can generate new dataset during active operations to support classifier update. Evaluation of the proposed framework is performed on an open dataset, i.e., ISCX VPN-nonVPN, and independently collected data packets. The results demonstrate that the proposed autonomous learning framework can filter packets from unknown classes and provide accurate labels. Thus, corresponding DL-based classification models can be updated successfully with the autonomously generated dataset.

9 days ago

4/10 relevant

arXiv

4/10 relevant

arXiv

Video Face Super-Resolution with Motion-Adaptive Feedback Cell

Video super-resolution (VSR) methods have recently achieved a remarkable success due to the development of deep convolutional neural

**networks**(CNN). Expand abstract. Video super-resolution (VSR) methods have recently achieved a remarkable success due to the development of deep convolutional neural

**networks**(CNN). Current state-of-the-art CNN methods usually treat the VSR problem as a large number of separate multi-frame super-resolution tasks, at which a batch of low resolution (LR) frames is utilized to generate a single high resolution (HR) frame, and running a slide window to select LR frames over the entire video would obtain a series of HR frames. However, duo to the complex temporal dependency between frames, with the number of LR input frames increase, the performance of the reconstructed HR frames become worse. The reason is in that these methods lack the ability to model complex temporal dependencies and hard to give an accurate motion estimation and compensation for VSR process. Which makes the performance degrade drastically when the motion in frames is complex. In this paper, we propose a Motion-Adaptive Feedback Cell (MAFC), a simple but effective block, which can efficiently capture the motion compensation and feed it back to the**network**in an adaptive way. Our approach efficiently utilizes the information of the inter-frame motion, the dependence of the**network**on motion estimation and compensation method can be avoid. In addition, benefiting from the excellent nature of MAFC, the**network**can achieve better performance in the case of extremely complex motion scenarios. Extensive evaluations and comparisons validate the strengths of our approach, and the experimental results demonstrated that the proposed framework is outperform the state-of-the-art methods.9 days ago

6/10 relevant

arXiv

6/10 relevant

arXiv

Handover Probability in Drone Cellular **Networks**

For the SSM, we compute the exact handover probability by establishing equivalence with a single-tier terrestrial cellular

**network**, in which the base stations (BSs) are static while the UEs are mobile. Expand abstract. This letter analyzes the handover probability in a drone cellular

**network**where the initial positions of drone base stations (DBSs) serving a set of user equipment (UE) on the ground are modeled by a homogeneous Poisson point process (PPP). Inspired by the mobility model considered in the third generation partnership project (3GPP) studies, we assume that all the DBSs move along straight lines in random directions. We further consider two different scenarios for the DBS speeds: (i) same speed model (SSM), and (ii) different speed model (DSM). Assuming nearest-neighbor association policy for the UEs on the ground, we characterize the handover probability of this**network**for both mobility scenarios. For the SSM, we compute the exact handover probability by establishing equivalence with a single-tier terrestrial cellular network, in which the base stations (BSs) are static while the UEs are mobile. We then derive a lower bound for the handover probability in the DSM by characterizing the evolution of the spatial distribution of the DBSs over time.9 days ago

5/10 relevant

arXiv

5/10 relevant

arXiv

Blind Adversarial **Network** Perturbations

However, deep neural

**networks**are known to be vulnerable to adversarial examples: adversarial inputs to the model that get labeled incorrectly by the model due to small adversarial perturbations. Expand abstract. Deep Neural

**Networks**(DNNs) are commonly used for various traffic analysis problems, such as website fingerprinting and flow correlation, as they outperform traditional (e.g., statistical) techniques by large margins. However, deep neural**networks**are known to be vulnerable to adversarial examples: adversarial inputs to the model that get labeled incorrectly by the model due to small adversarial perturbations. In this paper, for the first time, we show that an adversary can defeat DNN-based traffic analysis techniques by applying \emph{adversarial perturbations} on the patterns of \emph{live}**network**traffic.9 days ago

7/10 relevant

arXiv

7/10 relevant

arXiv

A closer look at the approximation capabilities of neural **networks**

The universal approximation theorem, in one of its most general versions, says that if we consider only continuous activation functions $\sigma$, then a standard feedforward neural network with one hidden layer is able to approximate any continuous multivariate function $f$ to any given approximation threshold $\varepsilon$, if and only if $\sigma$... Expand abstract.

The universal approximation theorem, in one of its most general versions, says that if we consider only continuous activation functions $\sigma$, then a standard feedforward neural

**network**with one hidden layer is able to approximate any continuous multivariate function $f$ to any given approximation threshold $\varepsilon$, if and only if $\sigma$ is non-polynomial. In this paper, we give a direct algebraic proof of the theorem. Furthermore we shall explicitly quantify the number of hidden units required for approximation. Specifically, if $X\subseteq \mathbb{R}^n$ is compact, then a neural**network**with $n$ input units, $m$ output units, and a single hidden layer with $\binom{n+d}{d}$ hidden units (independent of $m$ and $\varepsilon$), can uniformly approximate any polynomial function $f:X \to \mathbb{R}^m$ whose total degree is at most $d$ for each of its $m$ coordinate functions. In the general case that $f$ is any continuous function, we show there exists some $N\in \mathcal{O}(\varepsilon^{-n})$ (independent of $m$), such that $N$ hidden units would suffice to approximate $f$. We also show that this uniform approximation property (UAP) still holds even under seemingly strong conditions imposed on the weights. We highlight several consequences: (i) For any $\delta > 0$, the UAP still holds if we restrict all non-bias weights $w$ in the last layer to satisfy $|w| < \delta$. (ii) There exists some $\lambda>0$ (depending only on $f$ and $\sigma$), such that the UAP still holds if we restrict all non-bias weights $w$ in the first layer to satisfy $|w|>\lambda$. (iii) If the non-bias weights in the first layer are \emph{fixed} and randomly chosen from a suitable range, then the UAP holds with probability $1$.9 days ago

9/10 relevant

arXiv

9/10 relevant

arXiv

Coordinated Passive Beamforming for Distributed Intelligent Reflecting
Surfaces **Network**

However, current works mainly focus on single IRS-empowered wireless

**networks**, where the channel rank deficiency problem has emerged. Expand abstract. Intelligent reflecting surface (IRS) is a proposing technology in 6G to enhance the performance of wireless

**networks**by smartly reconfiguring the propagation environment with a large number of passive reflecting elements. However, current works mainly focus on single IRS-empowered wireless networks, where the channel rank deficiency problem has emerged. In this paper, we propose a distributed IRS-empowered communication**network**architecture, where multiple source-destination pairs communicate through multiple distributed IRSs. We further contribute to maximize the achievable sum-rates in this**network**via jointly optimizing the transmit power vector at the sources and the phase shift matrix with passive beamforming at all distributed IRSs. Unfortunately, this problem turns out to be non-convex and highly intractable, for which an alternating approach is developed via solving the resulting fractional programming problems alternatively. In particular, the closed-form expressions are proposed for coordinated passive beamforming at IRSs. The numerical results will demonstrate the algorithmic advantages and desirable performances of the distributed IRS-empowered communication**network**.10 days ago

7/10 relevant

arXiv

7/10 relevant

arXiv

Layered Embeddings for Amodal Instance Segmentation

Results demonstrate that the

**network**can accurately estimate complete masks in the presence of occlusion and outperform leading top-down bounding-box approaches. Expand abstract. The proposed method extends upon the representational output of semantic instance segmentation by explicitly including both visible and occluded parts. A fully convolutional

**network**is trained to produce consistent pixel-level embedding across two layers such that, when clustered, the results convey the full spatial extent and depth ordering of each instance. Results demonstrate that the**network**can accurately estimate complete masks in the presence of occlusion and outperform leading top-down bounding-box approaches. Source code available at https://github.com/yanfengliu/layered_embeddings10 days ago

4/10 relevant

arXiv

4/10 relevant

arXiv