`getroute` returns route even after channel has closed

[tony-voltage]

Issue and Steps to Reproduce

I don't know if it's exactly reproducible as is, but after opening a zero conf channel, making a payment, and then closing it, the channel continues to show up in get route well beyond it being closed (even after 100 blocks where it is "forgotten").

getinfo output

Here's the "receiver":

❯ ./lightning-cli --network regtest --lightning-dir=/Users/tony/.lightning2 --rpc-file=lightning-rpc2 getinfo
{
   "id": "03bacc7cc5292bf928e809b58cf5d61b3b7f60be130953d1f9b901c500fe8e305d",
   "alias": "LATENTTRAWL",
   "color": "03bacc",
   "num_peers": 1,
   "num_pending_channels": 0,
   "num_active_channels": 0,
   "num_inactive_channels": 0,
   "address": [
      {
         "type": "ipv4",
         "address": "0.0.0.0",
         "port": 18846
      }
   ],
   "binding": [
      {
         "type": "ipv4",
         "address": "0.0.0.0",
         "port": 18846
      }
   ],
   "version": "v0.12.1-1-g475c9bd",
   "blockheight": 302,
   "network": "regtest",
   "fees_collected_msat": 0,
   "lightning-dir": "/Users/tony/.lightning2/regtest",
   "our_features": {
      "init": "08a000080269a2",
      "node": "88a000080269a2",
      "channel": "",
      "invoice": "02000000024100"
   }
}

here's showing that there is no channels with the payer

❯ ./lightning-cli --network regtest --lightning-dir=/Users/tony/.lightning2 --rpc-file=lightning-rpc2 listpeers
{
   "peers": [
      {
         "id": "02a0ae3dfaf741ad5adc9be264cf0003e26998123c4c4de97938d1c32c15d29679",
         "connected": true,
         "netaddr": [
            "127.0.0.1:58233"
         ],
         "features": "08a000080269a2",
         "channels": []
      }
   ]
}

here's the payer getroute call:

❯ ./lightning-cli --network regtest getroute id=03bacc7cc5292bf928e809b58cf5d61b3b7f60be130953d1f9b901c500fe8e305d fromid=02a0ae3dfaf741ad5adc9be264cf0003e26998123c4c4de97938d1c32c15d29679 amount_msat=10000000 riskfactor=0 maxhops=1
{
   "route": [
      {
         "id": "03bacc7cc5292bf928e809b58cf5d61b3b7f60be130953d1f9b901c500fe8e305d",
         "channel": "13858584x8152606x12487",
         "direction": 0,
         "amount_msat": 10000000,
         "delay": 9,
         "style": "tlv"
      }
   ]
}

That channel existed previously, and after close (but before the 100 mined blocks after close), this is what it looked like from the payer side:

{
         "id": "03bacc7cc5292bf928e809b58cf5d61b3b7f60be130953d1f9b901c500fe8e305d",
         "connected": true,
         "netaddr": [
            "127.0.0.1:18846"
         ],
         "features": "08a000080269a2",
         "channels": [
            {
               "state": "ONCHAIN",
               "scratch_txid": "a91fc84de58f8c2909f4eb1a606a6df17e44c1be24d90b496084aa4db8177589",
               "last_tx_fee_msat": 170000,
               "feerate": {
                  "perkw": 253,
                  "perkb": 1012
               },
               "owner": "onchaind",
               "short_channel_id": "188x1x1",
               "direction": 0,
               "channel_id": "01ef7bf166ef4dd182a8ad42cd61a20d420f354f0b484f1fffa0612dc0ecde9c",
               "funding_txid": "9ddeecc02d61a0ff1f4f480b4f350f420da261cd42ada882d14def66f17bef01",
               "funding_outnum": 1,
               "close_to_addr": "bcrt1q7hmctqxfk2t93k3nrcphh749pfacz5sthudsjl",
               "close_to": "0014f5f78580c9b29658da331e037bfaa50a7b81520b",
               "private": true,
               "opener": "local",
               "closer": "remote",
               "alias": {
                  "local": "13858584x8152606x12487",
                  "remote": "722820x14594877x23379"
               },
               "features": [
                  "option_static_remotekey"
               ],
               "funding": {
                  "local_funds_msat": "20000000msat",
                  "remote_funds_msat": "0msat",
                  "pushed_msat": 0
               },
               "to_us_msat": 8000000,
               "min_to_us_msat": 8000000,
               "max_to_us_msat": 20000000,
               "total_msat": 20000000,
               "fee_base_msat": 1,
               "fee_proportional_millionths": 10,
               "dust_limit_msat": 546000,
               "max_total_htlc_in_msat": 18446744073709551615,
               "their_reserve_msat": 546000,
               "our_reserve_msat": 546000,
               "spendable_msat": 6914000,
               "receivable_msat": 11454000,
               "minimum_htlc_in_msat": 0,
               "minimum_htlc_out_msat": 0,
               "maximum_htlc_out_msat": 19454000,
               "their_to_self_delay": 6,
               "our_to_self_delay": 6,
               "max_accepted_htlcs": 483,
               "state_changes": [
                  {
                     "timestamp": "2023-01-05T17:12:58.159Z",
                     "old_state": "CHANNELD_AWAITING_LOCKIN",
                     "new_state": "CHANNELD_NORMAL",
                     "cause": "user",
                     "message": "Lockin complete"
                  },
                  {
                     "timestamp": "2023-01-05T18:19:26.356Z",
                     "old_state": "CHANNELD_NORMAL",
                     "new_state": "CHANNELD_SHUTTING_DOWN",
                     "cause": "remote",
                     "message": "Peer closes channel"
                  },
                  {
                     "timestamp": "2023-01-05T18:19:26.373Z",
                     "old_state": "CHANNELD_SHUTTING_DOWN",
                     "new_state": "CLOSINGD_SIGEXCHANGE",
                     "cause": "remote",
                     "message": "Start closingd"
                  },
                  {
                     "timestamp": "2023-01-05T18:19:26.383Z",
                     "old_state": "CLOSINGD_SIGEXCHANGE",
                     "new_state": "CLOSINGD_COMPLETE",
                     "cause": "remote",
                     "message": "Closing complete"
                  },
                  {
                     "timestamp": "2023-01-05T18:19:42.851Z",
                     "old_state": "CLOSINGD_COMPLETE",
                     "new_state": "FUNDING_SPEND_SEEN",
                     "cause": "remote",
                     "message": "Onchain funding spend"
                  },
                  {
                     "timestamp": "2023-01-05T18:19:42.873Z",
                     "old_state": "FUNDING_SPEND_SEEN",
                     "new_state": "ONCHAIN",
                     "cause": "remote",
                     "message": "Onchain init reply"
                  },
                  {
                     "timestamp": "2023-01-05T18:26:54.843Z",
                     "old_state": "ONCHAIN",
                     "new_state": "FUNDING_SPEND_SEEN",
                     "cause": "remote",
                     "message": "Onchain funding spend"
                  },
                  {
                     "timestamp": "2023-01-05T18:26:54.849Z",
                     "old_state": "FUNDING_SPEND_SEEN",
                     "new_state": "ONCHAIN",
                     "cause": "remote",
                     "message": "Onchain init reply"
                  },
                  {
                     "timestamp": "2023-01-05T18:27:12.138Z",
                     "old_state": "ONCHAIN",
                     "new_state": "FUNDING_SPEND_SEEN",
                     "cause": "remote",
                     "message": "Onchain funding spend"
                  },
                  {
                     "timestamp": "2023-01-05T18:27:12.145Z",
                     "old_state": "FUNDING_SPEND_SEEN",
                     "new_state": "ONCHAIN",
                     "cause": "remote",
                     "message": "Onchain init reply"
                  },
                  {
                     "timestamp": "2023-01-05T18:29:46.167Z",
                     "old_state": "ONCHAIN",
                     "new_state": "FUNDING_SPEND_SEEN",
                     "cause": "remote",
                     "message": "Onchain funding spend"
                  },
                  {
                     "timestamp": "2023-01-05T18:29:46.174Z",
                     "old_state": "FUNDING_SPEND_SEEN",
                     "new_state": "ONCHAIN",
                     "cause": "remote",
                     "message": "Onchain init reply"
                  },
                  {
                     "timestamp": "2023-01-05T18:30:02.637Z",
                     "old_state": "ONCHAIN",
                     "new_state": "FUNDING_SPEND_SEEN",
                     "cause": "remote",
                     "message": "Onchain funding spend"
                  },
                  {
                     "timestamp": "2023-01-05T18:30:02.644Z",
                     "old_state": "FUNDING_SPEND_SEEN",
                     "new_state": "ONCHAIN",
                     "cause": "remote",
                     "message": "Onchain init reply"
                  },
                  {
                     "timestamp": "2023-01-05T18:30:10.889Z",
                     "old_state": "ONCHAIN",
                     "new_state": "FUNDING_SPEND_SEEN",
                     "cause": "remote",
                     "message": "Onchain funding spend"
                  },
                  {
                     "timestamp": "2023-01-05T18:30:10.895Z",
                     "old_state": "FUNDING_SPEND_SEEN",
                     "new_state": "ONCHAIN",
                     "cause": "remote",
                     "message": "Onchain init reply"
                  },
                  {
                     "timestamp": "2023-01-05T18:31:22.070Z",
                     "old_state": "ONCHAIN",
                     "new_state": "FUNDING_SPEND_SEEN",
                     "cause": "remote",
                     "message": "Onchain funding spend"
                  },
                  {
                     "timestamp": "2023-01-05T18:31:22.076Z",
                     "old_state": "FUNDING_SPEND_SEEN",
                     "new_state": "ONCHAIN",
                     "cause": "remote",
                     "message": "Onchain init reply"
                  },
                  {
                     "timestamp": "2023-01-05T18:31:33.359Z",
                     "old_state": "ONCHAIN",
                     "new_state": "FUNDING_SPEND_SEEN",
                     "cause": "remote",
                     "message": "Onchain funding spend"
                  },
                  {
                     "timestamp": "2023-01-05T18:31:33.366Z",
                     "old_state": "FUNDING_SPEND_SEEN",
                     "new_state": "ONCHAIN",
                     "cause": "remote",
                     "message": "Onchain init reply"
                  }
               ],
               "status": [
                  "ONCHAIN:Tracking mutual close transaction",
                  "ONCHAIN:All outputs resolved: waiting 88 more blocks before forgetting channel"
               ],
               "in_payments_offered": 0,
               "in_offered_msat": 0,
               "in_payments_fulfilled": 0,
               "in_fulfilled_msat": 0,
               "out_payments_offered": 3,
               "out_offered_msat": 12000000,
               "out_payments_fulfilled": 3,
               "out_fulfilled_msat": 12000000,
               "htlcs": []
            },

Indicating it was mutually closed. I'll see if I can easily reproduce this, but currently it's pretty nasty for an LSP to think there's a route to the receiver when there's not. Could be related to other bugs around getroute : #5878

[cdecker]

This is a common misconception: getroute which is implemented in the topology plugin does not inspect the local state of the channels, and bases all of its decisions on the gossip messages stored in gossip_store. Therefore it'll be unaware of the channel's state and its balances.

What plugins like pay do is to create a local overlay on top of the graph, annotating channels with their learned constraints (learnt balances from attempts, and exact balances from listpeers) and use that to inform the getroute call (actually pay doesn't use getroute anymore, opting for a local dijkstra implementation), by excluding channels we know are unlikely to be usable due to their state or balances. This then results in those channels not being considered and topology not having to do the overlay tracking (which the caller will likely want to do itself as extending customization on that were the majority of requests).

[tony-voltage]

So it's normal that closed channels show up in getroute indefinitely?

[lsunsi]

@tony-voltage From my understanding, it would stop showing on getroute as soon as the channel is seen as closed by the topology layer via gossip? For example, if 1ML doesn't show the channel, it'd stop showing up on getroute, so to speak. Is this right @cdecker ?

[cdecker]

That's correct, as soon as we see the close onchain we add a tombstone entry to the gossip store, so the closed channel will stop showing up once the node sees the close confirm. Even if we don't see it close, the spec mandates that we should forget channels that havent been updated for 2w+, so a passive pruning mode is also in there.

You can assume that the gossip topology view + listpeers will always give you the best possible view of the network.

[tony-voltage]

That all sounds good and all, and I'm a bit indifferent to the underlying reasons why, all I know is that occasionally 0 conf channels never leave from getroute, at least on regtest. Whether or not you consider that "expected behavior" is up to you guys I guess. It's certainly not expected from my standpoint.

[harvhat]

I would have expected getroute to at least know about the local channel states and balances, as I think many people would, so it should at least be mentioned in the lightning-getroute(7) man page that this is not so.