This is correct; until someone on those remote instances searches (or maybe subscribes?) then the c/ultralight won’t be federated to those remote instances. You should use a larger community as your test case, if you are trying to verify federation, or take is a sign that your (very niche?) ultralight community just hasn’t got many subscribers yet.
Why are you browsing on a different instance? Stay on lemmy.world and search for communities from there. You should be able to see like http://lemmy.world/c/asklemmy@lemmy.ml. Note that you are on your instance but the @ tells it to grab the community from the other.