It is the nitrogen
that does the trick. No combustion can take place without
oxygen. The 4% limit refers to hydrogen in air which contains
oxygen. When the 5% hydrogen/nitrogen mix is released into the
air it dilutes the air. No matter how much you release, there
will either be too little hydrogen or too little oxygen to allow
the gas to ignite. Any mix of hydrogen and nitrogen containing
less than 5.7% hydrogen is classified as a non-flammable gas.
The safety margin is actually greater than that. Even a 10%
hydrogen/nitrogen mix is extremely difficult (but not
impossible) to ignite.