04-What-about-Gradient-Descent.txt

TITL:
   *What about Gradient Descent?*
   *By Forrest Landry*
   *Oct 7th and 11th, 2022*.

ABST:
   Some more question and answer dialogue,
   mostly about technical aspects related to
   learning and choice making.

   Also considers the meaning and nature
   of what would be required regulations.

TEXT:

   - where listing acronyms:.
     - SGD; as Stochastic Gradient Descent.
     - AGI; as Artificial General Intelligence.
     - APS; as Advanced Planning and Strategy Systems.

:ggu
   > In your essays, you do not particularly address
   > the selection effects produced by SGD,
   > which are *vastly* more powerful
   > than the selection effects produced by evolution.

     - that the fact that some effects are stronger
     in one sense, one domain, over one scope of time,
     does *not* mean that other effects
     are irrelevant in other scopes of time,
     other domains, or in other relevant abstractions.

:gk2
   > SGD deserves more attention
   > than simply speculating
   > that SGD might select for "code variants"
   > that are somehow "bad".

     - ?; who said anything about the mechanism
     of selection of such code variants
     being in any way defined by SGD at all?.

     - we may want to stop short of something like
     a presumption, assumption, or declaration
     of *both* of:.
       1; "everything is SGD", and then.
       2; "SGD does not do X".
     - that it ends up being something of a
     "presume what you want to prove" problem.
       - ie; has no relevance to our argument/claims.

:mrk1
   > That the odds of
   > a randomly sampled configuration of weights
   > being competent at self-reproduction
   > and thus of some kind of human subversion
   > are tiny --
   > far too unlikely to matter.

     - the replication process, at least initially,
     has absolutely nothing to do with such weights.
     - where at first; that *humans* are selecting
     which code AGI/APS variants get reproduced.
     - that they (the humans) mostly select
     on the basis of what succeeds in an environment,
     (ie; what is practical/profitable in a marketplace).

     - that we are simply noticing
     that the overall motion
     is one of, gradually,
     a shift in the environment
     in the favor of that AGI/APS replication capability.
       - ie; to make more, better, more automated factories,
       and then to notice what sort of conditions
       are needed within those factories
       as required to make more AGI/APS robots.

     - where/if we sight along the trend-lines
     just a little bit more;
     then, at some point,
     that the systems are selected
     for self repeatability --
     fully automated self production/reproduction,
     maybe even inclusive of self design/optimization
     to better both be profitable
     within the existing environment/context,
     and then; over time, to maybe shift
     the environment/context/marketplace
     to be one more allowing of profit extraction.

     - that we each notice, in this,
     that there is in that last aspect --
     a kind of _decoupling_,
     one that occurs in the overall economics
       (ie; in the operating context/environment)
     such that humans are more and more
     no longer a part of the economy.
       - that this is a trend that is important
       and has larger ramifications later.

     - where in the interim, however,
     we also notice that the AGI/APS
     design optimization process also
     has become fully automated,
     at which point, both direct environment feedback
     and maybe something of the SGD weights process
     might matter,
     if that is still relevant
     in the AGI design space --
     something we cannot now assume.
       - ?; does it become relevant
       that some neural network with SGD
       might have some implicit feedback
       to/towards artificiality benefit?.
       - as not obvious at all,
       at least at first (ie; today).

:gs8
   > - ?; are you assuming away value alignment
   > by characterizing the "true" goal
   > of the AGI internals
   > as self-replication?.

     - no; not affecting of overt goals,
     nor even implying it is a single "true goal",
     that would be displacing all other concepts;
     rather that it is an implicit intrinsic motivator,
     whether objectively stated/known/knowable or not.

:mrk2
   > Are you suggesting that we cannot, even in principle,
   > exert any kind of overriding selection pressure?
   > That argument does not make any sense at all.
   > People breed animals in all sorts of specific ways,
   > all of which end up 'overriding evolution'.

     Yes, we can/do implement 'selection pressure'
     in/with/within evolution process among animals
     by fully controlling/forcing their mate selections.
       - ie; by fully *denying* them (via full control)
       their own natural mate selection choices thereby.
       - as a kind of non-consensual slavery, etc (@ note 1 #note1).

     The total strength of 'the selection pressure'
     that we can exert (even under ideal circumstances)
     is actually only a very small fraction
     of what is actually going on --
     nature does nearly all of the work for us
       (ie; one forced macroscopic change is not
       equivalent to the billions of microscopic changes
       that occur in each moment in all of the trillions
       of cells that the creature is made of).
         - ie; that we can force mating patterns,
         yet we cannot create the animal children
         through some sort of manual process --
         we still need the animal code to do the
         animal process, in its own way.
       - that any genetic manipulation and
       "test tube baby process" we do
       is still very largely based on
       the code that nature had figured out
         (the hard way, by exhaustive 'try everything'
         and 'have absolutely zero subjective bias' --
         what nature/evolution automatically does,
         over the last billion years or so).
         - that we are good at making simple tweaks
         here and there, sometimes, in some ways,
         more or less like a hacker
         changing a few select lines of code,
         in an overall million line code base.
         - we can hardly call the hacker
         "the author" and/or the "developer"
         of "the whole system"
         when their total "contribution"
         is significantly less than 10^-10%.

:cl1
   > So, what is the overall claim?.

     - *if* (and only if/where) AGI gets developed;
     that there is no possible/implementable technique
     that could both detect and limit
     more than a tiny subset of
     all of the possible locally selected-for
     code variants
     that would (eventually) cause/compel
     accumulative changes to the environment --
     ones that are inherently out of line
     with conditions that humans need to survive.
       - that such variants
       are inherently more likely
       to be made and selected for,
       simply on the basis of
       practical existential necessity.

     Where/if self-learning, self-defining/making
     machine architectures are made to exist,
     and if/where/when they continue to exist;
     that it is categorically impossible
     to prevent/block all (but a tiny subset of) variants
     of the computed (appx) digital 'code'
     from also being selected (inherently)
     to be causing of outputs/outside effects
     which in interaction with
     the rest of the physical environment/context
     will lead those selected variants to replicate
     at a more frequent and sustained rate
     across the (global) environment."

       Ie; neither humans, nor even some *other* AGI,
       could account for and/or ultimately
       counteract/limit all the outcomes
       of the (random) fluctuations/relations
       in/of underlying physical substrate,
       along with all the chaotic dynamics
       (as entangled with all of planet Earth)
       in a way that would limit such selection.

:gww
   > Building AIs that can exchange code --
   > like bacteria exchange DNA --
   > would be Very Bad.

     Yes, and elsewhere we show
     that such code sharing
     is not, even in principle, preventable,
     if the notion of AGI means anything like
     "learning machine" or "adaptive".
       - cite (@ SuperIntelligence Safety https://mflb.com/ai_alignment_1/si_safety_qanda_out.html) for example.

:gy6
   > Maybe this could be made into a good outcome?
   > An AGI could store multiple copies of its past self,
   > which then can be used to veto any decisions
   > that are 'too contrary to their values' --
   > and/or intervene if the current AGI
   > seems to be malfunctioning somehow.

     - ?; is this somehow a proposal
     for an "error correcting protocol"?.

     - where if so; then it seems to completely
     conflate simple boolean fixed choice "voting"
     with *all* even slightly more complex/abstract
     choice making process --
     as if all "decision making processes" are strictly,
     or can somehow be made, strictly isomorphic with
     (ie; somehow equivalent to, in actual practice)
     all other abstract *objective* goal selection processes,
     inclusive of selection of basis of choice process (values),
     inclusive of "alignment with human interests"
     and/or "safe" with respect to carbon based life.

       - oh, wait --
       ?; are those deeper value/alignment choices
       somehow to be treated as immutable, fixed,
       and yet abstract, totally coherent, relevant,
       whereas every other level/process
       of choice/selection/decision
       is to be treated as "optimizeable",
       and therefore as "mutable"?.

       - where given the assumption that all choice
       processes are isomorphic with voting;
       and therefore scale invariant --
       equivalent at all levels of abstraction;.
       - ?; how is this assumption of
       "that we can achieve AGI alignment"
       /not/ a complete contradiction?.

:st1
   > Is the assumption you are disproving
   > maybe somehow also
   > an assumption that you introduced?.
   > If so, you defeating a strawman.

     We are actually making a specific claim --
     a kind of positively specified statement
     about a negative probability.
     What we are (here/elsewhere) claiming
     is that there are clear principles
     that show that it is *impossible*
     for any engineer, corporation, etc,
     to develop an AGI/APS system
     that in practice can *guarantee*
     that the actual likelihood of
     'a very bad thing happening'
     is at least strictly below
     some reasonable maximum allowable
     probability threshold --
     ie, one that reasonably reflects
     the consensus value of the combined
     result of the associated
     cost, benefit, risk tradeoff.

     The tacit assumption that we are disproving --
     the false hope that we are dispelling --
     is the idea that someone could
     maybe potentially eventually make
     and/or use some (*any*) type of AGI/APS system --
     even if done fully secretly and privately --
     and also have that action be,
     in any way at all, somehow "safe" for humanity,
     long term.

:h22
   > Even if you somehow 'disprove X',
   > how well does this reduce risks?.

     In the sense of people/researchers/corporations
     no longer trying to themselves do an impossible X,
     and/or not allowing others to implement a false X,
     then maybe a lot.

:h3l
   > Is there any way at all to do AGI/APS
   > that is not an x-risk, something that
   > at least moderates the problem a bit?.
   > Surely, there must be some exception.

     No, not that we can see.

     Either we actually fully account for
     the fact of the effects of artificiality
     over the long term,
     or we fail to do so,
     at our peril.

:h4u
   > What about all of the other work of
   > research into how to do AGI/APS?
   > Do you see any value and/or benefit
   > in our (failed) attempts to make AGI safe,
   > at least seemingly in the short term,
   > even if we cannot do so over the long term?.

     To the extent that these results,
     such as improved interpretability, clarity, etc,
     can be applied to *Narrow* single domain AI,
     then at the moment, aside from the inherent
     inequality issues, I see no problems with that.

     There is no question that NAI is useful.
     It just happens that it is also the case
     that AGI *cannot* be _actually_ useful.
     Distinguishing these two, in practice,
     is therefore paramount to our species.

     Even when regarding full NAI deployments,
     we should be careful, as engineers/developers,
     and keep the precautionary principle in mind.
     Ie, when working adjacent to an x-risk area,
     it is far better to move *extra* slowly,
     and be very sure to get the right answers,
     then it is to make some (silly) mistake
     and accidentally kill everything/everyone.

:h6q
   > So we must implement a perfect hard firewall
   > against a/any/the/all AGI/APS development?.
   > Inclusive of mass social commons sanctions
   > on anyone who even suggests slightly trying?.

     Basically, yes.

:safe1
   > Is there is an actionable
   > a solution to the problem
   > you have identified?

   > Maybe we should have the AI industry --
   > as with any new tech industry
   > which is developing new technology
   > that is increasingly closer
   > to uncontrollable --
   > to regulate itself
   > to the same extent
   > that the automotive industry
   > regulates itself
   > to so as to ensure
   > that it does not cross the line
   > of designing uncontrollable products?

   > Maybe we could require research
   > in even the NAI space to produce
   > automotive levels of safety docs?

     The auto industry standards,
     stiff as they are,
     are way too low.
     It is a bit more like regulating a bio lab
     to not do gain of function research
     (making *worse* pandemic bio weapons)
     without systemic safety protocols.

     Yet even that is not at all enough.
     For example, even really bad bio agents
     simply are not potentially permanently fatal
     to the entire planet, *forever*.

     When the severity indicators
     are fully orders of magnitude worse
     than even full all out WWW III nuclear exchange,
     any sort of proxy legal limits
     simply do not seem very realistic.
     Even a very low 0.00001% chance of death,
     in this case, of everyone,
     of total (all species) extinction,
     of everything that is wild and beautiful --
     of all life and aliveness lost for ever --
     is just not acceptable.

     Asking the industry to regulate itself,
     has historically been a non-start,
     even for relatively benign industries
     such as with aircraft safety.
     Consider, for example, what the FAA did
     with Boeing big-corp, and the result
     of its '737 MAX' product/system.

     As such, bio-labs simply should
     just not ever do *any* sort of
     'gain of function' research.
     Just not at all, not for any reason.
     Not ever, not even with the very best
     "systemic safety controls".
     They simply do not have the right
     to make those kinds of choices,
     and take those kinds of risks,
     ostensibly on behalf of everyone else.
     Even the highest security labs
     still fail in unacceptable simple ways,
     at unacceptably high rates,
     far too often,
     and we *all* pay the price.

     When considering it internationally,
     that asking biotech to self regulate
     seems like a clear false start.
     There are far too many smart idiots,
     paid for by naive govs, doing things
     that ought to never be done at all.

     It is not even clear if international
     nuclear treaty regulation levels
     are sufficient in this case.
     We might need something stronger still.

     We would need to prevent the existence
     of supercomputers, of *any* accumulation
     of total compute beyond certain limits,
     much the same way we carefully regulate
     the total amount of plutonium
     that can ever be in one place, at one time,
     in any/every country in the world.

     In the same way that any/every atomic pile
     is required to be internationally inspectable,
     we will need to somehow verifiably ensure
     that any 'oversized compute pile'
     is also inspectable, and/or simply
     have it be prevented from existing at all,
     beyond certain ranges --
     ie, say anything within even 10%
     of human equivalent brain capability.

     As we are starting to see
     with some actual anti-trust movement,
     that Big-Co Data-centers
     simply are not in the public interest.
     Especially as now, very publicly,
     we are starting to recognize
     that basically any form of
     total compute inequality
     really represents a total inequality
     in choice agency, of real power --
     via the very nature of code itself.

:note1

  - where remembering the 'people as pets' scenario,
  where some superintelligence decides to keep us,
  it is possible to consider what if the situation
  was reversed; as if something was breeding farmers:.

    - ?; I wonder if any farmer would be "ok" with
    some other alien mind 'forcing their choices'
    regarding whom they could (and/or could not) sleep with,
    fully including denying/depriving them
    ever having any lover(s) altogether, for life,
    if perhaps the AGI machine/system in some opaque way
    judged that their innermost code (their DNA, etc)
    was somehow "unfit"?.