Here's what i've been able to figure out about Dylan and the VU around this time, as gleaned from _Edie_ (the bio of Edie Sedgwick), Uptight_ (The Velvet Underground story by Gerard Malanga (who danced with the VU on the Exploding Plastic Inevitable) and Victor Bockris), _POPism_ (Andy Warhol's book on the 60s), and various taped interviews (specifically Nico on Australian radio in 1986 and John Cale in a BBC documentary).
Nico appeared in the film La Dolce Vita, which we can presume Dylan saw cause he refers to it in Motorpsycho Nitemare. In 1965, Dylan gives her an acetate of "I'll Keep It With Mine" for her to sing. She later moves from UK to New York, bringing the acetate with her. She ends up at the Factory around the time Warhol is recruiting the VU for the Exploding Plastic Inevitable multi-art project. According to reports in _Uptight_, the VU plays "I'll Keep It With Mine" in concert with Nico singing.
EDIE SEDGWICK, from a rich New England family, has also attached herself to the factory (acutally, she was there before Nico). She dances in the first EPI shows in late 65 and early 66, but leaves. According to _Edie_, she was having some sort of relationship with Bob Dylan, but he eventually went off and married Sara. That book also says (or i should say someone (BOb Neuwirth?), becaue the book consists exclusively of excerpts of taped interviews with various people who knew her) that the Blonde on Blonde album was written for her. (It's easy for me to believe that for songs like "Leopard-Skin Pillbox Hat", "Just Like a Woman", "One Of Us Must Know", and maybe "When You Go Your Way and I Go Mine, while I can also believe that "Sad-Eyed Lady of the Lowlands" was written for Sara, and Dylan went to Nashville to get away from New York. But I am speculating here, of course). (By the way, regarding the line in "Sara" about staying up nights in the Chelsea Hotel writing "Sad-Eyed Lady of the Lowlands", that hotel was a central location where many of the Factory art crowd lived, including Sedgwick and I think Nico, too). Anyway, Dylan dumped her (if indeed he ever took her), and Edie had a major affair with Bob Neuwirth, which you can read about in the book.
Meanwhile, Nico had a 5 year old son named Ari, fathered by French actor Alain Delon (?), and in the Australian interview, she mentions that Bob used to babysit Ari around this time.
Andy Warhol, in POPism, describes seeing Dylan at some parties and poetry readings in Manhattan. And in the WBAI interview, Dylan mentions that he likes the FUGS, who were also part of the Village art scene. (The Fugs memebers, the VU, and Allen Ginsberg all contributed to the East Village Other Electronic Newspaper of 1965). Plus Ed Sanders of the Fugs also published a poetry magazine before starting the Fugs. Ginsberg contributed to the mag, as did many of the Beat poets. I don't know if Dylan ever did.
Anyway, back to the VU. Lou studied poetry briefly at Syracuse under Delmore Schwazrtz. But he obviously also liked rock and roll and he liked Dylan enough to steal the line "I've got fever in my pockets, down to my shoes" for the song "Guess I'm Falling In Love" (finally released with words on the VU MCMXCIII live recordings, and also recorded in 1988 or so by Maureen Tucker, and also bootlegged from a 1967 performance). So I think we can say that Lou admired Dylan's writing.
JOHN CALE, however, is another story. In the South Bank documentary (1986 or 1987), he flatly states that what attracted him to Lou's lyrics was that they weren't "self-pitying", which he thought Dylan's lyrics were. (He also says he dislikes "Blowin' In the Wind" because the lyrics are all questions.) But he does admire Leonard Cohen (he's been performing "Hallelujah" regularly since the I'm Your Fan appearance), who in turn was reportedly an admirer of Dylan. John Cale has also done a number of theater-oriented collaborations with Bob Neuwirth, one of which is apparently soon to be released on CD.