I used to have to walk through this shrine to get to my house so not surprisingly I've spent a lot of time birding the shrine precinct and the hill behind it, seeing some really good birds.
Perhaps, because of that I'm unreasonably biased but I don't think seeing so many good birds is solely down to the time I put in, and for me this has to be one of the two the best sites around the city during migration. Mount Inari, which at 265m might call into question quite what qualifies as a mountain in Japan, is the last of a narrowing line of hills separating Kyoto from Yamashina and I can imagine this line could funnel migrants to this terminal point.
Unexpected migrants I've found here include Mugimaki Flycatcher, a pair of Yellow-rumped Flycatchers one spring, my only Broad-billed Roller in the city and Japanese Scops Owl. After coming across three Japanese Night Herons in the same spot in different years (Sept/Oct) it's almost an expected migrant but others that definitely qualify are Oriental Honey Buzzard, Grey-faced Buzzard, Ashy Minivet, Japanese Paradise Flycatcher, Siberian Thrush, Japanese Robin, Siberian Blue Robin none of which I've seen in city parks.
It's rather quite in summer but the old growth trees in the shrine precinct have had breeding Japanese Woodpecker and Brown Hawk Owl breed there too. This is the only place I've seen Giant Flying Squirrel in Kyoto. Higher up the mountain are breeding Ural Owls and Copper Pheasant used to be fairly easy to see but unfortunately they seem to have disappeared entirely.
Winter birding is okay but there's nothing that can't be seen elsewhere around Kyoto and often the hills further north have a bigger variety of birds. I sometimes see Grey Bunting under the trees near the shrine where there's little understorey and birds can be seen well but they aren't so regular here in my experience. There is an excellent location for them elsewhere on the mountain but the undergrowth is so thick it can be a long wait, Takaragaiike may be a better bet.
To get there both JR and Keihan have a station at Fushimi Inari, from either it's a short walk towards the shrine, just head towards the hill. You'll walk up a pedestrian approach road into the shrine, through the complex and into the woodland beyond. As you walk up the short approach road the resaurants on the left sell grilled sparrow. I've never met anyone who's tried them and most people I've asked find the idea rather off putting, but they must sell. In fact not all are Eurasian Tree Sparrows as advertised, while I've seen an old guy arrive with a large basket of Sparrows and seen feathers post plucking in the back street, I've also seen boxes marked produce of South America. Produce of Equador... Argentina... or anywhere else is one thing but "South America" is vague? Who knows what they're grilling or where from.
After looking round the old-growth area, there have been breeding Japanese Woodpecker here and I've seen Giant Flying Squirrel at night, keep heading uphill through the torri gates. You'll soon come to a pond by a small shrine, follow the main path left then right at the top of a flight of steps. After that just keep going up till you come to a "crossroads" at a view point and turn right continuing uphill. Eventually you'll come to the shrine complex marked as Inariyama on the map below. There are several such places along the route but you'll know this one as it's the first you'll leave heading downhill. Soon see an unpaved track heading off to the right. Take that track, which is where (hopefully) the best birding starts. If it doesn't at least you'll have lost the crowds and no longer have to peer between gaps in the torri gates! Follow this track soon turning sharply left rather than going down some steep, rough steps until you come to another junction. On the map below you'll be in the north/south dark green strip beyond which the green is distinctly lighter because of an absence of conifurs. Turn left or go straight on, the former only goes about 150m before offering another choice of direction, the latter will bring you to the same point but takes longer. You could do a loop there and return the same route. Alternatively turn left downhill in the direction of the city which will eventually bring you back into a built up area and past Tofukuji temple to a busier street at the bottom. Turn left and in about 300m Keihan Tobakaido station will be to your right. There are numerous other tracks you could take but it would be a good idea to have a map of the area. The loop route I mentioned is where all the Japanese Night Herons have turned up in autumn.
Don't go at New Year as about two million people visit the shrine in the first week of the year.