Bread flour has more protein content than all-purpose, which helps with gluten development. Some recipes call for it when an especially chewy and elastic crumb is desired.
You'll notice some difference in crumb and texture with breads made with each flour, but the flours can generally be substituted for one another. I tend to exclusively use AP flour, too! If anything, using bread flour will give you a heavier and denser loaf; if you're getting heavy, dense loaves with AP flour, then something else might be going on.