You can apply online, but it's only available in Maine, New Hampshire, Vermont, Massachusetts, Connecticut and Upstate New York.
The bank's checking account page has the checking account details. The Simply Free Checking has no minimum balance requirements or monthly maintenance fee. The minimum initial deposit is $25. The bank has three credit cards and all seem to have no annual fee. Note, with the credit card, it's very likely that there'll be a hard credit inquiry which will temporarily ding your credit score.
Credit for this find goes to Fire Finance.