Pharyngeal Phantasies?

As I was checking out this post at New Covenant, I thought about an question that I have been pondering.  Rusty's post responded of a post from Pharyngula, in which PZ Myers often gives us his evolutionary wisdom.  I was looking at this post by PZ, in which he asserts that there are "deep homologies" in the development of the parathyroid glands in land-dwelling animals to the gills of fish.

This reminded me of this article by PZ that asserts that the "gill slits" of the early embryo gives evidence for macroevolution.

Why should human embryos have tails and build their faces from the same embryonic foundation that fish use to build gills? These are questions that Wells fails to address, and his approach seems to be to try to argue instead that they do not exhibit similarities as embryos.  Yet they do. These similarities argue for common ancestry.

An admission: I did not like studying head and neck embryology in my training, and I did not know anyone that did.  As PZ's post states, most of the structure of the face developed form these "gill slits" or pharyngeal arches present in the embryo.  We were responsible for knowing which facial structure derived from which pharyngeal arch.

The easiest way to remember the embryonic derivation came from an interesting anatomical fact.  Each one of the pharyngeal arches are innervated by a different cranial nerve.  Since I already knew which nerve innervated the structures of the face, it was easy to use this knowledge to remember the embryonic derivation of any facial structure.  You can see that each arch uses a different cranial nerve here.  For instance, I would remember that the digastric muscle, which was innervated by both cranial nerve V (anterior belly from the first arch) and cranial nerve VII (posterior belly from the second arch) was derived from the first and second pharyngeal arches.  This is confirmed here.  You can even pretend to be a med student in gross anatomy and see these muscles in a cadaver here.  (and yes, that's what it really looks like).

I always thought this was interesting, and it seemed to make sense.  The derivatives of the pharyngeal arches were all associated with a different cranial nerve.  However, I had an interesting question. If humans build their face using "the same embryonic foundation that fish use to build gills", then we would expect that fish gills would be innervated in a similar (homologous) way as the human face.  In other words, it would use CN V (first arch), VII (2nd arch), IX (3rd arch), and X 4th and 6th arch to innervate the gills.  If all of these cranial nerves are not associated with fish gills, this would call into question that the pharyngeal arches are actual derivatives of fish gills.

Embryos of the fish also have cranial nerves associated with each of pharyngeal arches.  I've never dissected a fish in order to examine its cranial nerves, but this paper was a study in which certain cranial nerves were cut to examine the effect on fish respiration.  Here is a diagram of the cranial nerves on this fish:

As you can see, only cranial nerve X and IX innervate the gills.  Cranial nerves V and VII have nothing to do with the gills.

Question: if the structures of the human face evolved from gills in a fish, why are CN V and VII, derivatives of the first and second pharyngeal arches, not present in fish gills?