feat: fix bug in backpropagation.

Panadestein · Jan 1, 2025 · edb5968 · edb5968
1 parent b9f47fa
commit edb5968
Show file tree

Hide file tree

Showing 2 changed files with 17 additions and 23 deletions.
diff --git a/src/bqn/nn.bqn b/src/bqn/nn.bqn
@@ -2,21 +2,21 @@ Setplot‿Plot ← •Import "../bqn-utils/plots.bqn"
 U ← 0.5-˜•rand.Range⟜0
 M ← +˝∘×⎉1‿∞
 
-Minn ← {r‿l𝕊𝕩:
+Minn ← {rt‿ly𝕊𝕩:
   A‿DA ← ⟨1÷1+⋆∘-, ⊢×1⊸-⟩
-  BP ← {ts‿w𝕊𝕩:
-    do ← (-⟜ts×DA)⊢´𝕩
-    𝕩×⌜¨do∾(⊑do){d𝕊w‿ly: (DA ly)×d M w}`⋈¨´(1↓⌽)¨w‿𝕩
+  BP ← {ts‿we𝕊𝕩:
+    do ← <(-⟜ts×DA)⊢´𝕩
+    ⌽𝕩×⌜¨do∾do{d𝕊w‿z: (DA z)×d M˜⍉w}`⋈¨´(1↓⌽)¨we‿𝕩
   }
-  FP ← {𝕨𝕊b‿w: A¨b+𝕨M w}`⟜(⋈¨´)
-  nn ← 𝕩{fs‿ts𝕊b‿w: b⋈w-r× ts‿w BP fs FP𝕩}´˜(U⚇1-⟜1∘≠⋈·<˘2⊸↕)l
+  FP ← {z𝕊b‿we: A¨b+z M we}`⟜(⋈¨´)
+  nn ← 𝕩{fs‿ts𝕊bi‿we: bi⋈we-rt× ts‿we BP fs FP𝕩}´˜(U⚇1-⟜1∘≠⋈·<˘2⊸↕)ly
   E ⇐ ⊢´FP⟜nn  
 }
 
 n‿ri‿rf‿dr ← 1000‿2.8‿4‿0.001
 @ ⊣ td ← {𝕩(⊣∾(⊣×1⊸-×⊢)⍟n)¨0 •rand.Range˜≠𝕩} ↕∘⌈⌾((ri+dr×⊢)⁼)rf
 
-lm ← 0.01‿⟨1, 10, 1⟩ Minn td
-≠lm.E 2.9
+lm ← 0.01‿⟨1, 500, 1⟩ Minn td
+lm.E 2.85
 
 )r Setplot "scatter" ⋄ •Out¨ Plot´  m {ns←≠⊑𝕨 ⋄ (>𝕨)⋈˜∘‿ns⥊ns/𝕩} r
diff --git a/src/nn.org b/src/nn.org
@@ -55,14 +55,14 @@ namely plotting, random number generation, and matrix product:
 The original C implementation has 175 lines excluding the optimization loop. The BQN version has only 10.
 
 #+begin_src bqn :tangle ./bqn/nn.bqn
-  Minn ← {r‿l𝕊𝕩:
+  Minn ← {rt‿ly𝕊𝕩:
     A‿DA ← ⟨1÷1+⋆∘-, ⊢×1⊸-⟩
-    BP ← {ts‿w𝕊𝕩:
-      do ← (-⟜ts×DA)⊢´𝕩
-      𝕩×⌜¨do∾(⊑do){d𝕊w‿ly: (DA ly)×d M w}`⋈¨´(1↓⌽)¨w‿𝕩
+    BP ← {ts‿we𝕊𝕩:
+      do ← <(-⟜ts×DA)⊢´𝕩
+      ⌽𝕩×⌜¨do∾do{d𝕊w‿z: (DA z)×d M˜⍉w}`⋈¨´(1↓⌽)¨we‿𝕩
     }
-    FP ← {𝕨𝕊b‿w: A¨b+𝕨M w}`⟜(⋈¨´)
-    nn ← 𝕩{fs‿ts𝕊b‿w: b⋈w-r× ts‿w BP fs FP𝕩}´˜(U⚇1-⟜1∘≠⋈·<˘2⊸↕)l
+    FP ← {z𝕊b‿we: A¨b+z M we}`⟜(⋈¨´)
+    nn ← 𝕩{fs‿ts𝕊bi‿we: bi⋈we-rt× ts‿we BP fs FP𝕩}´˜(U⚇1-⟜1∘≠⋈·<˘2⊸↕)ly
     E ⇐ ⊢´FP⟜nn  
   }
 #+end_src
@@ -132,18 +132,12 @@ the context of neural networks[fn:5]. First let's generate some reference data:
 Then we can train =Minn=, and generate some approximated data with the perceptron:
 
 #+begin_src bqn :tangle ./bqn/nn.bqn
-  lm ← 0.01‿⟨1, 10, 1⟩ Minn td
-  ≠lm.E 2.9
+  lm ← 0.01‿⟨1, 500, 1⟩ Minn td
+  lm.E 2.85
 #+end_src
 
 #+RESULTS:
-: Error: Mapping: Expected equal shape prefix (1‿10 ≡ ≢𝕨, ⟨10⟩ ≡ ≢𝕩)
-: at   nn ← 𝕩{fs‿ts𝕊b‿w: b⋈w-r× ts‿w BP fs FP𝕩}´˜(U⚇1-⟜1∘≠⋈·<˘2⊸↕)l
-:                           ^
-: at   nn ← 𝕩{fs‿ts𝕊b‿w: b⋈w-r× ts‿w BP fs FP𝕩}´˜(U⚇1-⟜1∘≠⋈·<˘2⊸↕)l
-:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-: at lm ← 0.01‿⟨1, 10, 1⟩ Minn td
-:                         ^^^^
+: ⟨ 0.6426013683115244 ⟩
 
 Let’s see if we’ve gotten the numbers right after learning. But then again, what is a number that a man may know it[fn:6]...