As leader
- 1958: Blues for Dracula (Riverside Records)
- 1959: Drums Around the World: Philly Joe Jones Big Band Sounds (Riverside)
- 1959: Showcase (Riverside)
- 1960: Philly Joe’s Beat (Atlantic)
- 1961: Together! (Atlantic) – with Elvin Jones
- 1968: Mo’ Joe (Black Lion)
- 1977: Mean What You Say (Sonet)
- 1977: Philly Mignon (Galaxy)
- 1978: Drum Songs (Galaxy)
- 1979: Advance! (Galaxy)
- 1981: Octet (Marge)
- 1982: To Tadd with Love (Uptown)
As sideman
With Miles Davis
- Miles: The New Miles Davis Quintet (1956)
- Cookin’ with The Miles Davis Quintet (1956)
- Relaxin’ with The Miles Davis Quintet (1956)
- Workin’ with The Miles Davis Quintet (1956)
- Steamin’ with The Miles Davis Quintet (1956)
- ‘Round About Midnight (1957)
- Porgy and Bess (1958)
- Milestones (1958)
- Someday My Prince Will Come (1961)
With others
- Clifford Brown – Memorial Album (1953; Blue Note)
- J.R. Monterose – J. R. Monterose (1956; Blue Note)
- Sonny Rollins – Tenor Madness (1956; Riverside)
- Bennie Green– Bennie Green with Art Farmer (1956)
- Sonny Rollins – Newk’s Time (1957; Blue Note)
- John Coltrane – Blue Train (1957; Blue Note)
- Joe Castro – Mood Jazz (1957; Atlantic)
- Art Pepper – Art Pepper Meets the Rhythm Section (1957; Riverside)
- Clark Terry – In Orbit (1958; Riverside)
- Jimmy Smith – Softly as a Summer Breeze (1958; Blue Note)
- Bill Evans – Everybody Digs Bill Evans (1958; Riverside Records)
- Blue Mitchell – Big 6 (1958; Riverside)
- Sonny Clark – Cool Struttin’ (1958; Blue Note)
- Bud Powell – Time Waits (1958; Blue Note)
- Freddie Hubbard – Goin’ Up (1960; Blue Note) Hub Cap (1961; Blue Note) Here to Stay (1962; Blue Note)
- Dexter Gordon – Dexter Calling… (1961; Blue Note)
- Blue Mitchell – Smooth as the Wind (1961; Riverside)
- Hank Mobley – Hank (1957; Blue Note) Poppin’ (1957; Blue Note) Workout (1961; Blue Note) Another Workout (1961; Blue Note) No Room for Squares (1963; Blue Note)
- Milt Jackson & Wes Montgomery – Bags Meets Wes! (1962; Riverside)
- Bill Evans – California Here I Come (1967; Verve)
- Archie Shepp – Archie Shepp & Philly Joe Jones – (1969; America)
- Red Garland – Keystones! (1977; Xanadu Records)