| Date: | September 25, 2019 |
| Time: | Noon |
| Location: | BLOC628 |
| Speaker: | Weston Baines, Texas A&M |
| Title: | Deep neural network for source detection in 2D high noise emission problems |
| Abstract: | Source detection in high noise environments is crucial for
single-photon emission computed tomography (SPECT) medical imaging and
especially for homeland security applications. In the latter case, one
deals with detection of low emission nuclear sources in the presence of
significant background noise (with SNR < 0.01). Direction sensitivity is
needed to achieve this goal. Collimation, used for that purpose in
standard gamma-cameras is not an option. Instead, Compton cameras are
used. Backprojection methods enable detection in a random uniform
background. In most practical applications, however, the presence of
cargo violates this assumption and renders backprojection methods ineffective. A deep neural network is implemented for the task that
exhibits higher sensitivity and specificity than the backprojection
techniques in a low scattering case and works well when presence of
cargo makes those techniques fail.
This is joint work with P. Kuchment (Math) and J. Ragusa (Nuclear Eng.) |
|
| Date: | November 13, 2019 |
| Time: | Noon |
| Location: | BLOC 628 |
| Speaker: | Christoper Snyder, UT Austin |
| Title: | Combinatorial Complexity of Deep Networks: Think Weight Configurations, not Perturbations! |
| Abstract: | Did you know that (ReLU) Deep Neural Networks (DNNs) trained on linearly separable data are linear classifiers? While it is widely appreciated that some data assumptions are necessary to explain generalization in deep learning, we observe that very strong data assumptions induce regularity in gradient descent trained DNNs that is entirely combinatorial in nature. That is, strong constraints exist between the binary neuron states and binary output, which simplify the description of the classification map. We present a hierarchical decomposition of the DNN discrete classification map into logical (AND/OR) combinations of intermediate (True/False) classifiers of the input. Those classifiers that can not be further decomposed, called atoms, are (interpretable) linear classifiers. Taken together, we obtain a logical circuit with linear classifier inputs that computes the same label as the DNN. This circuit does not structurally resemble the network architecture, and it may require many fewer parameters, depending on the configuration of weights. In these cases, we obtain simultaneously an interpretation and generalization bound (for the original DNN), connecting two fronts which have historically been investigated separately. We study DNNs in simple, controlled settings, where we obtain superior generalization bounds despite using only combinatorial information (e.g. no margin information). On the MNIST dataset. We show that the learned, internal, logical computations correspond to semantically meaningful (unlabeled) categories that allow DNN descriptions in plain English. We improve the generalization of an already trained network by interpreting, diagnosing, and replacing components \textit{within} the logical circuit that is the DNN. |
|
| Date: | November 20, 2019 |
| Time: | Noon |
| Location: | BLOC 628 |
| Speaker: | David Rolnick, UPenn |
| Title: | Identifying Weights and Architectures of Unknown ReLU Networks |
| Abstract: | The output of a neural network depends on its parameters in a highly nonlinear way, and it is widely assumed that a network's parameters cannot be identified from its outputs. Here, we show that in many cases it is possible to reconstruct the architecture, weights, and biases of a deep ReLU network given the ability to query the network. ReLU networks are piecewise linear and the boundaries between pieces correspond to inputs for which one of the ReLUs switches between inactive and active states. Thus, first-layer ReLUs can be identified (up to sign and scaling) based on the orientation of their associated hyperplanes. Later-layer ReLU boundaries bend when they cross earlier-layer boundaries and the extent of bending reveals the weights between them. Our algorithm uses this to identify the units in the network and weights connecting them (up to isomorphism). The fact that considerable parts of deep networks can be identified from their outputs has implications for security, neuroscience, and our understanding of neural networks. Joint work with Konrad Körding. |
|
Menu