v1.2.0 (cryptosharks131#81)

BhaagBoseDK · Jul 10, 2022 · 1baebbd · 1baebbd
1 parent 0bd2b3c
commit 1baebbd
Show file tree

Hide file tree

Showing 16 changed files with 632 additions and 191 deletions.
diff --git a/README.md b/README.md
@@ -32,6 +32,9 @@ services:
 docker-compose down
 docker-compose build --no-cache
 docker-compose up -d
+
+# OPTIONAL: remove unused builds and objects
+docker system prune -f
 ```
 
 ## Manual Umbrel Installation (now available directly from the Umbrel app store)
@@ -69,6 +72,9 @@ networks:
 docker-compose down
 docker-compose build --no-cache
 docker-compose up -d
+
+# OPTIONAL: remove unused builds and objects
+docker system prune -f
 ```
 
 ## Manual Installation
@@ -87,7 +93,7 @@ Tip: If you plan to only use the development server, you will need to setup whit
 ### Step 2 - Setup Backend Data, Automated Rebalancing and HTLC Stream Data
 The files `jobs.py`, `rebalancer.py` and `htlc_stream.py` inside lndg/gui/ serve to update the backend database with the most up to date information, rebalance any channels based on your lndg dashboard settings and to listen for any failure events in your htlc stream. A refresh interval of at least 10-20 seconds is recommended for the `jobs.py` and `rebalancer.py` files for the best user experience.
 
-Recommend Setup With Supervisord or Systemd
+Recommend Setup With Supervisord (least setup) or Systemd (most compatible)
 1. Supervisord  
   a) Setup supervisord config `.venv/bin/python initialize.py -sd`  
   b) Install Supervisord `.venv/bin/pip install supervisor`  
@@ -134,12 +140,16 @@ When updating, follow the same steps as above. You will also need to restart the
 `sudo systemctl restart uwsgi.service`  
 
 ## Key Features
-### Suggests And Automates Fee Rates
-LNDg will make suggestions on an adjustment to the current set outbound fee rate for each channel. This uses historical payment and forwarding data over the last 7 days to drive suggestions.
+### Batch Opens
+You can use LNDg to batch open up to 10 channels at a time with a single transaction. This can help to signicantly reduce the channel open fees incurred when opening multiple channels.
 
-You may see another adjustment right after setting the new suggested fee rate on some channels. This is normal and you should wait ~24 hours before changing the fee rate again on any given channel.
+### Watch Tower Management
+You can use LNDg to add, monitor or remove watch towers from the lnd node.
 
-To allow LNDg to automatically adjust the fee rates on selected channels (no more than once every 24 hours) be sure that the AR-Autofees setting is set to `1` on the Advanced Settings page. You will also need to make sure individual channels are enabled on the Fee Rates page. You can view a log of AF changes by opening the Autofees tab.
+### Suggests Fee Rates
+LNDg will make suggestions on an adjustment to the current set outbound fee rate for each channel. This uses historical payment and forwarding data over the last 7 days to drive suggestions. You can use the Auto-Fees feature in order to automatically act upon the suggestions given.
+
+You may see another adjustment right after setting the new suggested fee rate on some channels. This is normal and you should wait ~24 hours before changing the fee rate again on any given channel.
 
 ### Suggests New Peers
 LNDg will make suggestions for new peers to open channels to based on your node's successful routing history.  
@@ -173,6 +183,24 @@ The following data can be accessed at the /api endpoint:
 ### Peer Reconnection
 LNDg will automatically try to resolve any channels that are seen as inactive, no more than every 3 minutes per peer.
 
+## Auto-Fees
+### Here are some additional notes to help you get started using Auto-Fees (AF).
+LNDg can update your fees on a channel every 24 hours if there is a suggestion listed on the fee rates page. You must make sure the `AF-Enabled` setting is set to `1` and that individual channels you want to be managed are also set to `enabled`. You can view a log of AF changes by opening the Autofees tab.
+
+You can customize some settings of AF by updating the following settings:  
+`AF-FailedHTLCs` - The minimum daily failed HTLCs count in which we could trigger a fee increase (depending on flow)  
+`AF-Increment` - The increment size of our potential fee changes, all fee suggestions will be a multiple of this value  
+`AF-MaxRate` - The maximum fee rate in which we can adjust to  
+`AF-MinRate` - The minimum fee rate in which we can adjust to  
+`AF-Multiplier` - Multiplier used against the flow pattern algorithm, the larger the multiplier, the larger the potential moves  
+
+AF Notes:
+1. AF changes only trigger after 24 hours of no fee updates via LNDg
+2. Single step maximum increase set to 15% or 25 ppm (which ever is increase smaller)
+3. Single step maximum decrease set to 25% or 50 ppm (which ever is decrease smaller)
+4. Channels with less than 25% outbound liquidty will not have their fees decreased
+5. Channels with less than 25% inbound liquidty will not have their fees increased
+
 ## Auto-Rebalancer - [Quick Start Guide](https://github.com/cryptosharks131/lndg/blob/master/quickstart.md)
 ### Here are some additional notes to help you better understand the Auto-Rebalancer (AR).
 
@@ -188,7 +216,12 @@ The objective of the Auto-Rebalancer is to "refill" the liquidity on the local s
 8. Enable `INBOUND` receving channels you would like to target and set an inbound liquidity `Target%` on the specific channel. Rebalance attempts will be made until inbound liquidity falls below this channel settting.
 9. The `INBOUND` receving channel is the channel that later routes out real payments and earns back the fees paid. Target channels that have lucrative outbound flows.
 10. Attempts that are successful or attempts with only incorrect payment information are tried again immediately. Example: If a rebalancing for 50k was sucessful, AR will try another 50k immediately with the same parameters.
-11. Attempts that fail for other reasons will not be tried again for 30 minutes after the stop time. This allows the liquidity in the network to move around for 30 mins before trying another rebalancing attempt that previously failed.
+11. Attempts that fail for other reasons will not be tried again for 30 minutes after the stop time. This allows the liquidity in the network to move around for 30 mins before trying another rebalancing attempt that previously failed. The 30 minute window can be customized by updating the `AR-WaitPeriod` setting.
+
+Additional customization options:
+1. `AR-Autopilot` - Automatically act upon suggestions on the AR Actions page
+2. `AR-WaitPeriod` - How long AR should wait before scheduling a channel that has recently failed an attempt
+3. `AR-Variance` - How much to randomly vary the target rebalance amount by this % of the target amount
 
 #### Steps to start the Auto-Rebalancer:
 1. Update Channel Specific Settings  
@@ -234,6 +267,12 @@ If you want a channel not to be picked for rebalancing (i.e. it is already full
 ![image](https://user-images.githubusercontent.com/38626122/137809882-4a87f86d-290c-456e-9606-ed669fd98561.png)
 ![image](https://user-images.githubusercontent.com/38626122/148699417-bd9fbb49-72f5-4c3f-811f-e18c990a06ba.png)
 
+### Manage Auto-Fees Or Get Suggestions
+![image](https://user-images.githubusercontent.com/38626122/175364451-a7e2bc62-71bd-4a2d-99f6-6a1f27e5999a.png)
+
+### Batch Open Channels
+![image](https://user-images.githubusercontent.com/38626122/175364599-ac894b68-a11d-420b-93b3-3ee8dffc857f.png)
+
 ### Suggests Peers To Open With and Rebalancer Actions To Take
 ![image](https://user-images.githubusercontent.com/38626122/148699445-88efeacd-3cfc-429c-91d8-3a52ee633195.png)
 ![image](https://user-images.githubusercontent.com/38626122/148699467-62ebbd7d-9f36-4707-88fd-62f2cc2a5506.png)

diff --git a/gui/forms.py b/gui/forms.py
@@ -81,6 +81,8 @@ class AutoRebalanceForm(forms.Form):
     fee_rate = forms.IntegerField(label='fee_rate', required=False)
     outbound_percent = forms.FloatField(label='outbound_percent', required=False)
     max_cost = forms.FloatField(label='max_cost', required=False)
+    variance = forms.IntegerField(label='variance', required=False)
+    wait_period = forms.IntegerField(label='wait_period', required=False)
     autopilot = forms.IntegerField(label='autopilot', required=False)
 
 updates_channel_codes = [

diff --git a/gui/migrations/0028_auto_20220620_1105.py b/gui/migrations/0028_auto_20220620_1105.py
@@ -0,0 +1,23 @@
+# Generated by Django 3.2.7 on 2022-06-20 11:05
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('gui', '0027_rebalancer_manual'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='pendinghtlcs',
+            name='forwarding_channel',
+            field=models.CharField(max_length=20),
+        ),
+        migrations.AlterField(
+            model_name='rebalancer',
+            name='fee_limit',
+            field=models.FloatField(),
+        ),
+    ]
diff --git a/gui/migrations/0029_update_percent_vars.py b/gui/migrations/0029_update_percent_vars.py
@@ -0,0 +1,48 @@
+# Generated by Django 3.2.7 on 2022-06-27 08:47
+from django.db import migrations
+
+def update_percent_vars(apps, schedma_editor):
+    settings = apps.get_model('gui', 'localsettings')
+    try:
+        if settings.objects.filter(key='AR-MaxCost%').exists():
+            current_value = settings.objects.filter(key='AR-MaxCost%')[0]
+            current_value.value = int(float(current_value.value)*100)
+            current_value.save()
+        if settings.objects.filter(key='AR-Outbound%').exists():
+            current_value = settings.objects.filter(key='AR-Outbound%')[0]
+            current_value.value = int(float(current_value.value)*100)
+            current_value.save()
+        if settings.objects.filter(key='AR-Target%').exists():
+            current_value = settings.objects.filter(key='AR-Target%')[0]
+            current_value.value = int(float(current_value.value)*100)
+            current_value.save()
+    except Exception as e:
+        print('Migration step failed:', str(e))
+
+def revert_percent_vars(apps, schedma_editor):
+    settings = apps.get_model('gui', 'localsettings')
+    try:
+        if settings.objects.filter(key='AR-MaxCost%').exists():
+            current_value = settings.objects.filter(key='AR-MaxCost%')[0]
+            current_value.value = int(current_value.value)/100
+            current_value.save()
+        if settings.objects.filter(key='AR-Outbound%').exists():
+            current_value = settings.objects.filter(key='AR-Outbound%')[0]
+            current_value.value = int(current_value.value)/100
+            current_value.save()
+        if settings.objects.filter(key='AR-Target%').exists():
+            current_value = settings.objects.filter(key='AR-Target%')[0]
+            current_value.value = int(current_value.value)/100
+            current_value.save()
+    except Exception as e:
+        print('Migration reversion step failed:', str(e))
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('gui', '0028_auto_20220620_1105'),
+    ]
+
+    operations = [
+        migrations.RunPython(update_percent_vars, revert_percent_vars),
+    ]
diff --git a/gui/models.py b/gui/models.py
@@ -104,25 +104,25 @@ class Channels(models.Model):
     def save(self, *args, **kwargs):
         if not self.ar_out_target:
             if LocalSettings.objects.filter(key='AR-Outbound%').exists():
-                outbound_setting = float(LocalSettings.objects.filter(key='AR-Outbound%')[0].value)
+                outbound_setting = int(LocalSettings.objects.filter(key='AR-Outbound%')[0].value)
             else:
-                LocalSettings(key='AR-Outbound%', value='0.75').save()
-                outbound_setting = 0.75
-            self.ar_out_target = int(outbound_setting * 100)
+                LocalSettings(key='AR-Outbound%', value='75').save()
+                outbound_setting = 75
+            self.ar_out_target = int(outbound_setting)
         if not self.ar_amt_target:
             if LocalSettings.objects.filter(key='AR-Target%').exists():
-                amt_setting = float(LocalSettings.objects.filter(key='AR-Target%')[0].value)
+                amt_setting = int(LocalSettings.objects.filter(key='AR-Target%')[0].value)
             else:
-                LocalSettings(key='AR-Target%', value='0.05').save()
-                amt_setting = 0.05
-            self.ar_amt_target = int(amt_setting * self.capacity)
+                LocalSettings(key='AR-Target%', value='5').save()
+                amt_setting = 5
+            self.ar_amt_target = int((amt_setting/100) * self.capacity)
         if not self.ar_max_cost:
             if LocalSettings.objects.filter(key='AR-MaxCost%').exists():
-                cost_setting = float(LocalSettings.objects.filter(key='AR-MaxCost%')[0].value)
+                cost_setting = int(LocalSettings.objects.filter(key='AR-MaxCost%')[0].value)
             else:
-                LocalSettings(key='AR-MaxCost%', value='0.65').save()
-                cost_setting = 0.65
-            self.ar_max_cost = int(cost_setting * 100)
+                LocalSettings(key='AR-MaxCost%', value='65').save()
+                cost_setting = 65
+            self.ar_max_cost = int(cost_setting)
         super(Channels, self).save(*args, **kwargs)
 
     class Meta:
@@ -143,7 +143,7 @@ class Meta:
 class Rebalancer(models.Model):
     requested = models.DateTimeField(default=timezone.now)
     value = models.IntegerField()
-    fee_limit = models.IntegerField()
+    fee_limit = models.FloatField()
     outgoing_chan_ids = models.TextField(default='[]')
     last_hop_pubkey = models.CharField(default='', max_length=66)
     target_alias = models.CharField(default='', max_length=32)
@@ -209,7 +209,7 @@ class PendingHTLCs(models.Model):
     amount = models.BigIntegerField()
     hash_lock = models.CharField(max_length=64)
     expiration_height = models.IntegerField()
-    forwarding_channel = models.IntegerField()
+    forwarding_channel = models.CharField(max_length=20)
     forwarding_alias = models.CharField(max_length=32)
     class Meta:
         app_label = 'gui'

diff --git a/gui/templates/advanced.html b/gui/templates/advanced.html
@@ -138,31 +138,31 @@ <h2>Advanced Channel Settings</h2>
         <td>
           <form action="/update_channel/" method="post">
             {% csrf_token %}
-            <input style="text-align:center" id="target" type="number" min="0" max="100000000" name="target" value="{{ channel.ar_amt_target }}">
+            <input style="text-align:center" id="target" type="number" min="1" max="100000000" name="target" value="{{ channel.ar_amt_target }}">
             <input type="hidden" name="chan_id" value="{{ channel.chan_id }}">
             <input type="hidden" name="update_target" value="2">
           </form>
         </td>
         <td>
           <form action="/update_channel/" method="post">
             {% csrf_token %}
-            <input style="text-align:center" id="target" type="number" min="0" max="100" name="target" value="{{ channel.ar_max_cost }}">
+            <input style="text-align:center" id="target" type="number" min="1" max="100" name="target" value="{{ channel.ar_max_cost }}">
             <input type="hidden" name="chan_id" value="{{ channel.chan_id }}">
             <input type="hidden" name="update_target" value="6">
           </form>
         </td>
         <td {% if channel.auto_rebalance == False %}style="background-color: #80ced6"{% else %}style="background-color: #f18973"{% endif %}>
             <form action="/update_channel/" method="post">
               {% csrf_token %}
-              <input style="text-align:center" id="target" type="number" min="0" max="100" name="target" value="{{ channel.ar_out_target }}">
+              <input style="text-align:center" id="target" type="number" min="1" max="100" name="target" value="{{ channel.ar_out_target }}">
               <input type="hidden" name="chan_id" value="{{ channel.chan_id }}">
               <input type="hidden" name="update_target" value="4">
             </form>
         </td>
         <td {% if channel.auto_rebalance == True %}style="background-color: #80ced6"{% else %}style="background-color: #f18973"{% endif %}>
             <form action="/update_channel/" method="post">
               {% csrf_token %}
-              <input style="text-align:center" id="target" type="number" min="0" max="100" name="target" value="{{ channel.ar_in_target }}">
+              <input style="text-align:center" id="target" type="number" min="1" max="100" name="target" value="{{ channel.ar_in_target }}">
               <input type="hidden" name="chan_id" value="{{ channel.chan_id }}">
               <input type="hidden" name="update_target" value="3">
             </form>
@@ -202,18 +202,20 @@ <h2>Update Local Settings</h2>
       <td>
         <form action="/update_setting/" method="post">
           {% csrf_token %}
-          {% if settings.key|slice:"-1:" == '%' %}
-            <input style="text-align:center" id="value" type="number" step="0.01" min="0" max="1" name="value" value="{{ settings.value }}">
+          {% if settings.key == 'AR-Target%' %}
+            <input style="text-align:center" id="value" type="number" step="0.1" min="0.1" max="100" name="value" value="{{ settings.value }}">
+          {% elif settings.key|slice:"-1:" == '%' or settings.key == 'AR-Variance' or settings.key == 'AF-Increment' or settings.key == 'AF-Multiplier' or settings.key == 'AF-FailedHTLCs' or settings.key == 'AR-WaitPeriod'%}
+          <input style="text-align:center" id="value" type="number" min="1" max="100" name="value" value="{{ settings.value }}">
           {% elif settings.key == 'AR-Time' %}
-            <input style="text-align:center" id="value" type="number" min="0" max="60" name="value" value="{{ settings.value }}">
-          {% elif settings.key == 'AR-MaxFeeRate' %}
-            <input style="text-align:center" id="value" type="number" min="0" max="2500" name="value" value="{{ settings.value }}">
-          {% elif settings.key == 'LND-CleanPayments' %}
+            <input style="text-align:center" id="value" type="number" min="1" max="60" name="value" value="{{ settings.value }}">
+          {% elif settings.key == 'AR-MaxFeeRate' or settings.key == 'AF-MaxRate' %}
+            <input style="text-align:center" id="value" type="number" min="1" max="2500" name="value" value="{{ settings.value }}">
+          {% elif settings.key == 'AF-MaxRate' or settings.key == 'AF-MinRate' %}
+            <input style="text-align:center" id="value" type="number" min="0" max="5000" name="value" value="{{ settings.value }}">
+          {% elif settings.key == 'LND-CleanPayments' or settings.key|slice:":3" == 'AR-' or settings.key|slice:":3" == 'AF-'%}
             <input style="text-align:center" id="value" type="number" min="0" max="1" name="value" value="{{ settings.value }}">
           {% elif settings.key == 'LND-RetentionDays' %}
             <input style="text-align:center" id="value" type="number" min="0" max="1000" name="value" value="{{ settings.value }}">
-          {% elif settings.key|slice:":3" == 'AR-' or settings.key|slice:":3" == 'AF-' %}
-            <input style="text-align:center" id="value" type="number" min="0" max="1" name="value" value="{{ settings.value }}">
           {% else %}
             <input style="text-align:center" id="value" type="text" name="value" value="{{ settings.value }}">
           {% endif %}

diff --git a/gui/templates/base.html b/gui/templates/base.html
@@ -28,7 +28,7 @@ <h1><a href="/">My Lnd Overview</a></h1>
   <footer>
     <div id="footer">
       <div class="w3-container w3-padding-small">
-        <center>LNDg v1.1.2</center>
+        <center>LNDg v1.2.0</center>
       </div>
     </div>
   </footer>

diff --git a/gui/templates/channels.html b/gui/templates/channels.html
@@ -20,11 +20,11 @@ <h2>Channel Performance</h2>
         <th>Peer Alias</th>
         <th width=1%>Capacity</th>
         <th title="7-Day Routed Out | Rebalanced In">Outbound Flow</th>
-        <th title="7-Day APY">APY | CV</th>
+        <th title="7-Day APY | CV (Channel Value: annualized revenue + assisted revenue">APY | CV</th>
         <th title="7-Day Revenue Out [Net Profits] | Assisted Revenue In">Out [Profit] | In</th>
         <th title="7-Day Routed In | Rebalanced Out">Inbound Flow</th>
         <th title="30-Day Routed Out | Rebalanced In">Outbound Flow</th>
-        <th title="30-Day APY">APY | CV</th>
+        <th title="30-Day APY | CV (Channel Value: annualized revenue + assisted revenue">APY | CV</th>
         <th title="30-Day Revenue Out [Net Profits] | Assisted Revenue In">Out [Profit] | In</th>
         <th title="30-Day Routed In | Rebalanced Out">Inbound Flow</th>
         <th title="The count of updates is directly correlated to the space the channel takes up in your channel database." width=1%>Updates</th>