All quotes come from r/askhistorians comment by User druidofdarrowdelf, but it doesn't answer my own questions.
The Japanese captured a Navajo POW, Joe Kieyoomia, who wasn't part of the Code Talker program. He was forced to listen to the transmissions by his captors. The Japanese had studied the Navajo transmissions and had narrowed it down through a lengthy process (I can't understand Japanese so can't read the primary sources. Sorry for the vagueness), to figure out it was Navajo. Kieyoomia listened into these transmissions and heard phrases like "Red soil ahead" among all of the organizational information that was also coded. He thought it was complete gibberish, and told the Japanese that it made no sense. The Japanese thought he was lying, and tortured him regularly to extract more information about the Navajo language and code out of him. I don't feel as though Kieyoomia's resistance to Japanese efforts led the Japanese to believe that it really was just the language, and such limited their code breaking efforts there.
The code itself was simple, a glorified slang with a weird syntax, with applications for the military alphabet. The CIA provides an example of ship names. In English or any other language this could be cracked with time. Imagine a modern US infantry squad calling for "Apache support". To someone without knowledge of US weapon systems, this would appear a strange request. However after a few times hearing that and having helicopter gunships show up, the link could be drawn and part of the code could be cracked.
Then why couldn't the Japanese use Joe Kieyoomia's translation to infer the causation between Navajo words and their "Message or True Reading", and thus crack the code? For example, it feels straight forward and common sense that "the Navajo did not have a word for submarine so they translated it to iron fish".
I don't know if Japanese Intelligence Community was shambles. The Army/Navy rivalry was replicated between the different organizations, resulting in no real clear unified efforts and quite a few efforts to steal glory from each other. There were more focused on American operation codes that were transmitted in text, far more regularly and often and contained more information. But this all feels irrelevant, when the Japanese appeared to have gained headway into the Navajo Code already.
Research on the Navajo language
Navajo is one of the most difficult languages to learn. Its language family is the Athabaskan language. FEW non-Navajo have been nearly fluent. Navajo heavily relies on inflections of tone and use of nasal noises on vowels, so simple changes in pitch of a vowel mean a completely different word.
Navajo is being preserved better now due to many programs and educational outreach. But in 1942 Navajo was a dying language and there were FEW texts on it. You couldn't just pick books on Navajo off the shelf.
Research on the verbal use and speed of the Navajo Code Talkers
The Navajo Code was a tactical, and very rarely strategic code that was only submitted verbally. The Navajo Code was never used in a written form. If it had been, it could have been subjected to the same methods of code breaking that the world was using on operational and strategic codes like Enigma. The Navajo code was far less complex than Enigma and would not have held up well to such attacks. Audio recordings on wax cylinders were expensive and difficult to maintain in the Pacific theater, so even recording the radio messages sent by Code Talkers was incredibly difficult.
The Navajo Code was not only used on a tactical level, but on a much smaller scale than US operational codes. When the Code Talkers program was under development, time was its major selling point, not its heavy encryption. The current way for US forces to communicate on a open level was either "in the clear", i.e. just over the radio, or to use an encoding machine that usually took about an hour to encode and decode messages. The former the Japanese heavily exploited with their high percentage of fluent English speakers who gathered intelligence on the US and "sent out bogus messages in American code to lure marines into ambushes". The latter wasn't great when you needed fire support.
On the other hand Code Talkers could communicate messages to other Code Talkers nearly instantly. Fire support, movements, positions, could all be transmitted nearly "in the clear" in terms of speed. Also, Code Talkers worked like a sort of passcode, or key. With Navajo being so complex and the Code Talkers being such a small group, they recognized and knew each other during transmissions. And once attached units also recognized this, Code Talkers messages were treated as critically important, the Japanese couldn't falsely transmit them. Code Talkers were able to let everyone know when the Japanese sent out false messages being as if they were American. This uniqueness kept the code in the military arsenal (and the Navajo language was classified) until 1965, when frequency hopping radios started to become available and replaced the Code Talkers concept.