ProgramTestContext::set_account does not update programs

[andreisilviudragnea]

Problem

When calling ProgramTestContext::set_account twice for the same program address: &Pubkey, the second call gets ignored, so the account data will always be the one set on the first call. The following test illustrates the problem:

    #[tokio::test]
    async fn test_upgradeable_program_account_set_program_data_account_address_works() {
        let program_id = Pubkey::new_unique();

        let mut context = ProgramTest::default().start_with_context().await;

        let program_data_address = Pubkey::new_unique();
        context.set_account(
            &program_id,
            &upgradeable_program_account(program_data_address),
        );

        context.set_account(
            &program_data_address,
            &program_data_account("helloworld1.so", 0),
        );

        let result = simulate_transaction(&mut context, program_id).await;
        assert_eq!(
            result.simulation_details.unwrap().logs[1],
            "Program log: Hello World Rust program entrypoint 1"
        );

        context.warp_to_slot(2).unwrap();

        let program_data_address = Pubkey::new_unique();
        context.set_account(
            &program_id,
            &upgradeable_program_account(program_data_address),
        );
        context.set_account(
            &program_data_address,
            &program_data_account("helloworld0.so", 2),
        );

        context.warp_to_slot(3).unwrap();

        let result = simulate_transaction(&mut context, program_id).await;
        assert_eq!(
            result.simulation_details.unwrap().logs[1],
            "Program log: Hello World Rust program entrypoint 1" // TODO should be 0
        );
    }

    fn upgradeable_program_account(program_data_address: Pubkey) -> AccountSharedData {
        let account_len = UpgradeableLoaderState::size_of_program();

        let mut account = Account {
            lamports: Rent::default().minimum_balance(account_len).max(1),
            data: vec![0; account_len],
            owner: bpf_loader_upgradeable::id(),
            executable: true,
            rent_epoch: 0,
        };

        account
            .set_state(&UpgradeableLoaderState::Program {
                programdata_address: program_data_address,
            })
            .unwrap();

        AccountSharedData::from(account)
    }

    fn program_data_account(path: &str, slot: Slot) -> AccountSharedData {
        let program_data = read_file(path);

        let program_data_len =
            UpgradeableLoaderState::size_of_programdata_metadata() + program_data.len();

        let mut program_data_account = Account {
            lamports: Rent::default().minimum_balance(program_data_len).max(1),
            data: vec![0; program_data_len],
            owner: bpf_loader_upgradeable::id(),
            executable: true,
            rent_epoch: 0,
        };

        program_data_account
            .set_state(&UpgradeableLoaderState::ProgramData {
                slot,
                upgrade_authority_address: None,
            })
            .unwrap();

        program_data_account.data[UpgradeableLoaderState::size_of_programdata_metadata()..]
            .copy_from_slice(&program_data);

        AccountSharedData::from(program_data_account)
    }

    async fn simulate_transaction(
        context: &mut ProgramTestContext,
        program_id: Pubkey,
    ) -> solana_banks_interface::BanksTransactionResultWithSimulation {
        let tx = Transaction::new_signed_with_payer(
            &[Instruction::new_with_bytes(
                program_id,
                &[],
                vec![AccountMeta::new_readonly(program_id, false)],
            )],
            Some(&context.payer.pubkey()),
            &[&context.payer],
            context.last_blockhash,
        );

        context.banks_client.simulate_transaction(tx).await.unwrap()
    }

I reproduced this bug here.

Proposed Solution

I expect the second call to ProgramTestContext::set_account to update the state of the program account. I proposed a solution in #34780.

I reproduced this bug on Solana v1.16.25 and v1.17.15.

[Lichtso]

The problem is that you are updating the program account directly, where as you should be using a deployment instruction of the loader. If you don't you will continue to see the outdated version which is cached.