According to Wikipedia, there is a penalty of $20 per day until the form is filed (capped at the minimum of $1000 and 5% of the gross receipts of the organization for that year). If the gross receipts are more than $1 million, the penalty is increased to $100 per day (capped at $50,000). The penalty for failure to provide to the public is $20 per day. There are other penalties for e.g. omitting information.
Public Inspection IRC 6104(d) regulations state that an organization must provide copies of its three most recent Forms 990 to anyone who requests them, whether in person, by mail, fax, or e-mail.