# Neural Network - Switch Net with back-propagation

**URL:** <https://discourse.processing.org/t/neural-network-switch-net-with-back-propagation/41892>\
**Category:** Gallery\
**Created:** [May 3, 2023, 2:11am UTC](https://discourse.processing.org/t/neural-network-switch-net-with-back-propagation/41892 "2023-05-03T02:11:48Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![seanc87](https://avatars.discourse-cdn.com/v4/letter/s/b5e925/32.png) [@seanc87](https://discourse.processing.org/u/seanc87)\
**Post date:** [May 3, 2023, 2:11am UTC](https://discourse.processing.org/t/neural-network-switch-net-with-back-propagation/41892/1 "2023-05-03T02:11:48Z")

</div>

It is kind of difficult to implement wide neural networks on a CPU because the amount of compute required scales according to width by width.  
A dense neural layer of width 256 needs 256 by 256 fused multiply adds, and it just gets worse from there.  
Using Switch Net the real limit is the L2 cache size of the CPU you are using. Beyond that the fast transform used starts to slow down too much as it repeatedly has to fecth data off the DRAM chips rather than internal CPU memory. Within that you are sort of getting GPU type performance from a CPU.  
I think it is worth experimenting with, you might find a use for it.

Here is a sketch with the Switch Net neural network code:  
[https://editor.p5js.org/siobhan.491/sketches/RvqZfikaE](https://editor.p5js.org/siobhan.491/sketches/RvqZfikaE)

At the end of this blog post about the subject, in the comments, there is a link to some regular (Java) processing code: [https://ai462qqq.blogspot.com/2023/04/switch-net.html](https://ai462qqq.blogspot.com/2023/04/switch-net.html)

---

<div class="post-metadata">

**Author:** ![seanc87](https://avatars.discourse-cdn.com/v4/letter/s/b5e925/32.png) [@seanc87](https://discourse.processing.org/u/seanc87)\
**Post date:** [May 9, 2023, 2:43am UTC](https://discourse.processing.org/t/neural-network-switch-net-with-back-propagation/41892/2 "2023-05-09T02:43:52Z")

</div>

I see there is not much interest here, oh, well.  
Here is Switch Net 4 in JavaScript:  
[https://editor.p5js.org/siobhan.491/sketches/sHU0dNmWk](https://editor.p5js.org/siobhan.491/sketches/sHU0dNmWk)  
Perhaps I should point out you can train extremely wide neural networks using such algorithms that would be impossible using JavaScript and conventional dense neural network layers. Anyway I’ll post it as a matter of reference.