There is a reasonable level of usability, that normal users expect and that business wise developers provide to keep a customer base. Failure to provide this level of support is taken at the site owner's risk.
Owing to many reasons of the normal usability needs of disabled users are forgotten when developers make accommodations. Some times these accomodations provide a good level of physical accessibly from the perceptability or physical operability point of view. However, these accomodations abandon minimal usability convensions.
So disabled users are frequently faced with the cruel dilema of physical access for the price of minimal usability.
Perhaps the point is this. One cannot claim accessiblity if its presence reduces the level of usability below reasonable levels.
So, one cannot talk about accessiblity without metioning usability for at least one reason. The implementation of accessibility cannot introduce unusability.
There are probably many other factors that link the two concepts. Perhaps we should identify the key reasons explicitly.