It is interesting to read a book like Chomsky’s Syntactic Structures while thinking about thinking. To what extent does the structure of thought mirror the structure of language? Does thought have a grammar in the way languages have a grammar? Does the apparatus of linguistic theory carry over to thinking? Are there psychic structures parallel to syntactic structures?
There are two sorts of reason for supposing that linguistic and psychic structure might march in parallel, both familiar. First, syntax has “psychological reality”: it is an aspect of linguistic competence—what the speaker or hearer must grasp and process. The grammar of a language is reflected in the understanding of that language. So a theory of grammar is a theory of a psychological capacity. Second, every sentence of a language like English expresses a potential thought: whatever you can say you can think (more or less). All of the resources of grammar are available to thought: it isn’t that thought sits in an impoverished corner of the mind while language flaunts its wealth elsewhere. For any sentence S of a language L there is a thought T such that speakers of L can have T. For example, a verb like take can assume many forms, e.g. takes, has taken, will take, has been taken, is being taken, etc (the example is from Syntactic Structures, p. 38). Sentences formed from these so-called auxiliary verbs correspond to a range of thoughts that a normal human thinker is capable of forming, such as the thought that the cat has been taken to the vet. This is not the same as the thought that the cat will be taken to the vet. Thus a normal thinker can have any thought expressed in the language of which he is a master. But this means that the thinker must have the conceptual resources to think such thoughts; and that includes structural resources—the mental analogue of grammar. There must be at least as much psychic structure as there is syntactic structure. Indeed, one might be forgiven for supposing these structures to be, at a deep level, identical.
If this is so, there is a branch of psychology that mirrors linguistics. The same issues that arise about language can arise about thought. I will list a few of these. There is the question of what kind of grammar applies to both: is it a Markov process grammar, a phrase structure grammar, a transformational grammar, or something else? There is the question of how finite resources can generate an infinite number of strings (sentences or thoughts—strings of concepts). There is the question of acquisition: how is the capacity to have all these thoughts acquired by the child? How much is innate and how much learned? Are there psychic universals as there are linguistic universals? Is there a competence-performance distinction for thoughts? What is the abstract structure of a theory of thought? How much of the machinery of thought is unconscious? Are there rules of thought construction as there are rules of sentence construction? In both language and thought we have a capacity to combine elements into complex wholes, and the capacities are clearly related: so both raise the same kinds of explanatory questions that will receive similar kinds of answers. It is true that language takes the form of external sounds while thinking is inner and silent, but this is surely a superficial difference—the structure of both capacities is the same or similar. 
Thus Chomsky could have written a companion volume Psychic Structures and pursued many of the same questions at the level of thinking; indeed, we might suppose that he has already written that volume in writing Syntactic Structures. All we need to do is change some of the descriptive vocabulary and Chomsky will be interpretable as writing about thought not language. Now that cognitive science has liberated itself from behaviorist strictures we can directly pursue the question of psychic structure; and maybe that was our topic all along. 
 Invoking a language of thought will effect an immediate unification: the structure of thoughts will be a syntactic structure in the head. Then Chomsky could be taken to be writing about syntactic structures in the language of thought, as well as external public languages. But we need not presuppose such a theory in order to appreciate the theoretical parallels between language and thought.