Under what circumstances can dispatched fighter jets shoot down an aircraft full of alive passengers?

When the pilots are obeying a lawful order of a superior commissioned officer

Under 10 U.S. Code § 890 - Art. 90. it is an offence to disobey the lawful command of a superior commissioned officer.

Of course, unlawfully killing a human being is murder under 10 U.S. Code § 918, so it turns on the pilot’s knowledge of the lawfulness or unlawfulness of the order.

A superior’s order is presumed to be lawful and is disobeyed at the subordinate’s peril. To sustain the presumption, the order must relate to military duty. It must not conflict with the statutory or constitutional rights of the person receiving the order. Finally, it must be a specific mandate to do or not to do a specific act. In sum, an order is presumed lawful if it has a valid military purpose and is a clear, precise, narrowly drawn mandate. United States v. Moore, 58 M.J. 466 (C.A.A.F. 2003). The dictates of a person’s conscience, religion, or personal philosophy cannot excuse disobedience. United States v. Stockman, 17 M.J. 530 (A.C.M.R. 1973).

If the pilots are given a clear order to shoot down the aircraft from a superior, their actions will almost certainly be lawful. They are not required to independently determine the military necessity of the action; they are required to carry it out.

In a court martial, the lawfulness or otherwise of an order is a matter of law for the judge to determine, not a matter of fact for the panel. As such , the determination is something that can be appealed.

Lawfulness of the originating order

Since September 11, 2001, the US Air Force has primary responsibility for all US air space. The exact rules of engagement that they do this under are, for obvious reasons, not public.

It is known that flying into restricted areas without following the protocols, failing to communicate, or failing to follow instructions are all things that could lead to a plane being lawfully shot down.

It is believed that the authority to issue such an order is reserved to people high up in the chain of command. Possibly to politicians such as the President, the Secretary of Defence and the Deputy Secretary of Defence, although it’s also possible that serving officers above a certain rank in the relevant chain of command may also have the authority.

If the order came from the President, the Supreme Court has recently told us in USA v Trump, that the President cannot be criminally liable for anything done in their official capacity - such as giving orders to the military. So, while it may be illegal for the President to order the shooting down of the a regularity scheduled British Airways New York to London flight on a whim, they can’t be prosecuted for it. If the President gave the order to a serving officer in the chain of command, it would be a lawful order.

Anyone else making the decision does not enjoy that level of immunity, so they would have to demonstrate that they followed the rules of engagement and that their decision to kill the civilians was a proportionate military necessity.

What legislation is there on this

None AFAIK. This is a common law power of the executive branch and Congress has decided not to constrain the executive’s decision.

International law

Customary international law has norms, while not universally accepted, that a country does not have a right to fire on foreign civilian aircraft in its airspace. Of course, any country can blow up as many of its own aircraft as it wants.

It is probably only legitimated once the situation has developed to the point where the aircraft poses a real and credible threat to the hot nation’s security, triggering the right to self-defence under article 51 of the UN charter.

That said, no one has ever been prosecuted in an international court for shooting down a civilian aircraft.