[FF7, Field, PSX, Animation] BCX/BSX file formats?

[Micky]

I'm trying to get BCX/BSX files displayed in my own viewer. I found my original code to read the mesh, but it looks like I didn't get animations to work. Searching here I found several people's decoding attempts, but most of the links are dead now. Does anyone still have his notes, or can update the Wiki with the information?
By the way, where in q-gears is the code to extract the information from the original disks? The SVN version I have seems to load the extracted data directly.

[Akari]

I added converter to SVN. Check it out.

[Micky]

Great, thank you! I've got it now.

[Micky]

Just found another bit: the byte after the vertex count in each part seems to be the number of texcoords.

[Micky]

After a bit of more exploration it looks like the unused byte after each vertex colour could be the normal index. The index has similar values as the vertex index (i.e. if two vertices have the same indices, the corresponding unused values are the same as well), but the normal table doesn't seem to be in the model. Maybe it is shared across all models and stored in the executable?
Another bit: The index goes from 0 to 240 for most BCX. Only Red 13 has a few less.
It's of course possible as well that the value directly encodes the value, and is split into two angle values.

[Akari]

After a bit of more exploration it looks like the unused byte after each vertex colour could be the normal index. The index has similar values as the vertex index (i.e. if two vertices have the same indices, the corresponding unused values are the same as well), but the normal table doesn't seem to be in the model. Maybe it is shared across all models and stored in the executable?
Another bit: The index goes from 0 to 240 for most BCX. Only Red 13 has a few less.
It's of course possible as well that the value directly encodes the value, and is split into two angle values.

I can't see if normals used at all. Data set into GPU packets directly. I continuing reversing data related to initialization of field models. Stay tuned and look into SVN =)

By the way did you found something about additional file in texture data in BSX?
I found it reading in asm, but cant find file where it used.

It used in in texture data if 5-7 bytes are not zeros (this is offset to something). 4th Byte are number of textures.

[Micky]

I can't see if normals used at all. Data set into GPU packets directly. I continuing reversing data related to initialization of field models. Stay tuned and look into SVN =)

I think the main character is "full bright" most of the time, but I'm sure I've seen shading in some of the areas where you see the characters large. But it would explain why I can't find a normal table. I'll have to double-check tonight. I can't imagine why they would add normals and lighting information to the PC version when it didn't exist in the PSX version.
At the moment I regenerate the normals based on face normals and shared vertices.

I'll try to get my code more stable and release it. at the moment I seem to run into some special cases when decoding BSX files. (For example vertex count being 0.)

[Akari]

I can't see if normals used at all. Data set into GPU packets directly. I continuing reversing data related to initialization of field models. Stay tuned and look into SVN =)

I think the main character is "full bright" most of the time, but I'm sure I've seen shading in some of the areas where you see the characters large. But it would explain why I can't find a normal table. I'll have to double-check tonight. I can't imagine why they would add normals and lighting information to the PC version when it didn't exist in the PSX version.
At the moment I regenerate the normals based on face normals and shared vertices.

I'll try to get my code more stable and release it. at the moment I seem to run into some special cases when decoding BSX files. (For example vertex count being 0.)

Lighting information calculated during model initialization and stored into GPU draft packets. It can be changed via KAWAI opcode which is recalculate lighting for all vertexes and store it to drafts.

Additional part in texture can be used to change global eye texture (FIELD.TDB). You can change image, palette. There is option to store given texture to anywhere in VRAM directly. And there is one more option which I can't understand. mystery solved (mostly) =)

[Micky]

Okay, for your amusement, a BCX/BSX viewer in Python:

Code: [Select]

import sys, struct, array, math, os, zlib

from OpenGL.GL import *
from OpenGL.GLU import *
from OpenGL.GL.ARB.texture_rectangle import *
from OpenGL.GL.ARB.multitexture import *

import pygame
from pygame.locals import *

bcxMap = {}
normals = []

def vecDot(a,b):
    return a[0]*b[0]+a[1]*b[1]+a[2]*b[2]

def vecNormal(a):
    lenSqr = vecDot(a,a)
    if lenSqr > 0.0:
ool = 1.0 / math.sqrt(lenSqr)
return (a[0] * ool, a[1] * ool, a[2] * ool)
    else:
return (0,0,0)

def loadLzs(name):
    # read compressed data
    f = open(name, "rb")
    buf = f.read()
    f.close()
    
    (size,) = struct.unpack_from("<I", buf, 0)
    if size + 4 != len(buf):
del buf
return None

    ibuf = array.array("B", buf)
    obuf = array.array("B")
    
    # decompress;
    iofs = 4
    cmd = 0
    bit = 0
    while iofs < len(ibuf):
if bit == 0:
   cmd = ibuf[iofs]
   bit = 8
   iofs += 1
if cmd & 1:
   obuf.append(ibuf[iofs])
   iofs += 1
else:
   a = ibuf[iofs]
   iofs += 1
   b = ibuf[iofs]
   iofs += 1
   o = a | ((b & 0xF0) << 4)
   l = (b & 0xF) + 3;

   rofs = len(obuf) - ((len(obuf) - 18 - o) & 0xFFF)
   for j in xrange(l):
if rofs < 0:
   obuf.append(0)
else:
   obuf.append(obuf[rofs])
rofs += 1
cmd >>= 1
bit -= 1

    return obuf.tostring()

def loadFieldBin(name):
    # read compressed data
    f = open(name, "rb")
    buf = f.read()
    f.close()
    
    (size,) = struct.unpack_from("<I", buf, 0)

    # decompress
    data = zlib.decompress(buf[8:], 16 | 15)
    if len(data) != size:
del buf
del data
return None
    return data
  
def readModel(data, num_bones, offset_skeleton, num_parts, offset_parts):
    model = {}
    
    skeleton = []
    for i in xrange(num_bones):
(length, parent, has_mesh) = struct.unpack_from("<hbB", data, offset_skeleton + i * 4)
skeleton.append((length, parent, has_mesh != 0))
    model["skeleton"] = skeleton
    
    parts = []
    for i in xrange(num_parts):
part = {}
(u1, bone_index, num_vertex, num_texcoord, num_quad_color_tex, num_tri_color_tex, num_quad_mono_tex, num_tri_mono_tex, num_tri_mono, num_quad_mono, num_tri_color, num_quad_color, u3, offset_poly, offset_texcoord, offset_flags, offset_control, u4, offset_vertex) = struct.unpack_from("<BBBBBBBBBBBBHHHHHHH", data, offset_parts + i * 0x20)
# print (u1, bone_index, num_vertex, num_texcoord, num_quad_color_tex, num_tri_color_tex, num_quad_mono_tex, num_tri_mono_tex, num_tri_mono, num_quad_mono, num_tri_color, num_quad_color, u3, offset_poly, offset_texcoord, offset_flags, offset_control, u4, offset_vertex)
#print "offset_vertex %x %x %x" % (offset_vertex, num_vertex, offset_vertex + 4 + num_vertex * 8)
#print "offset_texcoord %x %x %x" % (offset_vertex + offset_texcoord, num_texcoord, offset_vertex + offset_texcoord + num_texcoord * 2)
part["bone_index"] = bone_index
# print (u1, u3, u4)

vertices = []
for j in xrange(num_vertex):
   (x, y, z) = struct.unpack_from("<hhh", data, offset_vertex + 4 + j * 8)
   vertices.append((x,y,z))
part["vertices"] = vertices

texcoords = []
for j in xrange(num_texcoord):
   (u, v) = struct.unpack_from("<BB", data, offset_vertex + offset_texcoord + j * 2)
   texcoords.append((u,v))
part["texcoords"] = texcoords

cur_poly = offset_vertex + offset_poly
cur_flags = offset_vertex + offset_flags
cur_control = offset_vertex + offset_control

# print "u4 : %4.4x" % (offset_vertex + u4)
# print "cur_poly : %4.4x" % (cur_poly)
# print "cur_flags : %4.4x" % (cur_flags)
# print "cur_control : %4.4x" % (cur_control)

quad_color_tex = []
for j in xrange(num_quad_color_tex):
   control = struct.unpack_from("<B", data, cur_control)
   cur_control += 1

   (av, bv, cv, dv, ar, ag, ab, an, br, bg, bb, bn, cr, cg, cb, cn, dr, dg, db, dn, at, bt, ct, dt) = struct.unpack_from("<BBBBBBBBBBBBBBBBBBBBBBBB", data, cur_poly)
   poly = ((av, at, (ar, ag, ab), an), (bv, bt, (br, bg, bb), bn), (dv, dt, (dr, dg, db), dn), (cv, ct, (cr, cg, cb), cn))

   # print "quad_color_tex %i %i %i %i / %i %i %i %i" % (av, bv, cv, dv, an, bn, cn, dn)

   quad_color_tex.append(poly)
   cur_poly += 0x18
part["quad_color_tex"] = quad_color_tex

tri_color_tex = []
for j in xrange(num_tri_color_tex):
   control = struct.unpack_from("<B", data, cur_control)
   cur_control += 1

   (av, bv, cv, xv, ar, ag, ab, an, br, bg, bb, bn, cr, cg, cb, cn, at, bt, ct, xt) = struct.unpack_from("<BBBBBBBBBBBBBBBBBBBB", data, cur_poly)
   poly = ((av, at, (ar, ag, ab), an), (bv, bt, (br, bg, bb), bn), (cv, ct, (cr, cg, cb), cn), xv, xt)
   # print "tri_color_tex %i %i %i / %x %x %x" % (av, bv, cv, an, bn, cn)

   tri_color_tex.append(poly)
   cur_poly += 0x14
part["tri_color_tex"] = tri_color_tex

quad_mono_tex = []
for j in xrange(num_quad_mono_tex):
   control = struct.unpack_from("<B", data, cur_control)
   cur_control += 1

   (av, bv, cv, dv, r, g, b, n, at, bt, ct, dt) = struct.unpack_from("<BBBBBBBBBBBB", data, cur_poly)
   # print "quad_mono_tex", x
   poly = ((av, at), (bv, bt), (dv, dt), (cv, ct), (r, g, b), n)
   quad_mono_tex.append(poly)
   cur_poly += 0x0C
part["quad_mono_tex"] = quad_mono_tex

tri_mono_tex = []
for j in xrange(num_tri_mono_tex):
   control = struct.unpack_from("<B", data, cur_control)
   cur_control += 1

   (av, bv, cv, xv, r, g, b, n, at, bt, ct, xt) = struct.unpack_from("<BBBBBBBBBBBB", data, cur_poly)
   poly = ((av, at), (bv, bt), (cv, ct), (r, g, b), n, xv, xt)
   # print "tri_mono_tex", x, xv, xt
   tri_mono_tex.append(poly)
   cur_poly += 0x0C
part["tri_mono_tex"] = tri_mono_tex

tri_mono = []
for j in xrange(num_tri_mono):
   (av, bv, cv, xv, r, g, b, n) = struct.unpack_from("<BBBBBBBB", data, cur_poly)
   poly = ((av,), (bv,), (cv,), (r, g, b), n, xv)
   # print "tri_mono", x, xv
   tri_mono.append(poly)
   cur_poly += 8
part["tri_mono"] = tri_mono

quad_mono = []
for j in xrange(num_quad_mono):
   (av, bv, cv, dv, r, g, b, n) = struct.unpack_from("<BBBBBBBB", data, cur_poly)
   poly = ((av,), (bv,), (dv,), (cv,), (r, g, b), n)
   # print "quad_mono", x
   quad_mono.append(poly)
   cur_poly += 8
part["quad_mono"] = quad_mono

tri_color = []
for j in xrange(num_tri_color):
   (av, bv, cv, xv, ar, ag, ab, an, br, bg, bb, bn, cr, cg, cb, cn) = struct.unpack_from("<BBBBBBBBBBBBBBBB", data, cur_poly)
   poly = ((av,(ar, ag, ab), an), (bv, (br, bg, bb), bn), (cv, (cr, cg, cb), cn), xv)
   # print "tri_color %i %i %i / %i %i %i %i / %i" % (av, bv, cv, an, bn, cn, xv, len(vertices))

   tri_color.append(poly)
   cur_poly += 0x10
part["tri_color"] = tri_color

quad_color = []
for j in xrange(num_quad_color):
   (av, bv, cv, dv, ar, ag, ab, an, br, bg, bb, bn, cr, cg, cb, cn, dr, dg, db, dn) = struct.unpack_from("<BBBBBBBBBBBBBBBBBBBB", data, cur_poly)
   poly = ((av,(ar, ag, ab), an), (bv, (br, bg, bb), bn), (dv, (dr, dg, db), dn), (cv, (cr, cg, cb), cn))

   # print "quad_color %i %i %i %i / %x %x %x %x" % (av, bv, cv, dv, an, bn, cn, dn)

   quad_color.append(poly)
   cur_poly += 0x14
part["quad_color"] = quad_color

# print "end_poly : %4.4x" % (cur_poly)
# print "end_flags : %4.4x" % (cur_flags)
# print "end_control : %4.4x" % (cur_control)

parts.append(part)

    model["parts"] = parts
    
    return model
    
def readAnimations(data, num_animations, offset_animations):
    animations = []
    for i in xrange(num_animations):
(num_frames, num_channel, num_frames_translation, num_static_translation, num_frames_rotation, offset_frames_translation, offset_static_translation, offset_frames_rotation, offset_data) = struct.unpack_from("<HBBBBHHHI", data, offset_animations + i * 0x10)
offset_data &= 0x7FFFFFFF
# print "animation:", (offset_animations + i * 0x10, num_frames, num_channel, num_frames_translation, num_static_translation, num_frames_rotation, offset_frames_translation, offset_static_translation, offset_frames_rotation, offset_data)

animation = []
for k in xrange(num_channel):
   (flag, rx, ry, rz, tx, ty, tz) = struct.unpack_from("<BBBBBBB", data, offset_data + 0x04 + k * 8)

   channel = []
   for j in xrange(num_frames):

                # rotation
                if flag & 0x01:
                    rotation_x = 360.0 * float(ord(data[offset_data + offset_frames_rotation + rx * num_frames + j])) / 255.0
                else:
                    rotation_x = 360.0 * rx / 255.0

                if flag & 0x02:
                    rotation_y = 360.0 * float(ord(data[offset_data + offset_frames_rotation + ry * num_frames + j])) / 255.0
                else:
                    rotation_y = 360.0 * ry / 255.0

                if flag & 0x04:
                    rotation_z = 360.0 * float(ord(data[offset_data + offset_frames_rotation + rz * num_frames + j])) / 255.0
                else:
                    rotation_z = 360.0 * rz / 255.0

                # translation
                if flag & 0x10:
                    (translation_x,) = struct.unpack_from("<h", data, offset_data + offset_frames_translation + tx * num_frames * 2 + j * 2)
                elif tx != 0xFF:
                    (translation_x,) = struct.unpack_from("<h", data, offset_data + offset_static_translation + tx * 2)
else:
   translation_x = 0

                if flag & 0x20:
                    (translation_y,) = struct.unpack_from("<h", data, offset_data + offset_frames_translation + ty * num_frames * 2 + j * 2);
                elif ty != 0xFF:
                    (translation_y,) = struct.unpack_from("<h", data, offset_data + offset_static_translation + ty * 2)
else:
   translation_y = 0

                if flag & 0x40:
                    (translation_z,) = struct.unpack_from("<h", data, offset_data + offset_frames_translation + tz * num_frames * 2 + j * 2)
                elif tz != 0xFF:
                    (translation_z,) = struct.unpack_from("<h", data, offset_data + offset_static_translation + tz * 2)
else:
   translation_z = 0

channel.append(((translation_x, translation_y, translation_z), (rotation_x, rotation_y, rotation_z)))

   animation.append(channel)
animations.append(animation)

    return animations

def loadBcx(name):
    data = loadLzs(name)
    (size, header_offset) = struct.unpack_from("<II", data, 0)
    assert(size == len(data))
    (u1, num_bones, num_parts, num_animations, u2, u3, u4, offset_skeleton) = struct.unpack_from("<HBBB19sHHI", data, header_offset)
    offset_skeleton &= 0x7FFFFFFF
    offset_parts = offset_skeleton + num_bones * 4
    offset_animations = offset_parts + num_parts * 32
    # print "header", (header_offset, u1, num_bones, num_parts, num_animations, u2, u3, u4, offset_skeleton, offset_parts, offset_animations)

    object = {}
    object["model"] = readModel(data, num_bones, offset_skeleton, num_parts, offset_parts)
    object["animations"] = readAnimations(data, num_animations, offset_animations)
    
    return object
    
def loadBsx(name):
    # get compressed file
    data = loadLzs(name)
    (size, header_offset) = struct.unpack_from("<II", data, 0)
    assert(size == len(data))

    # decode individual objects
    object_list = []
    (u1, num_models) = struct.unpack_from("<II", data, header_offset)
    for i in xrange(num_models):
model_offset = header_offset + 0x10 + i * 0x30
(model_id,u2,u3,offset_skeleton,u4,u5,u6,u7,u8,u9,u10,u11,u12,u13,u14,u15,u16,u17,u18,u19,u20,num_bones,u22,u23,u24,u25,u26,u27,u28,u29,u30,u31,u32,num_parts,u34,u35,u36,u37,u38,u39,u40,u41,u42,u43,u44,num_animations) = struct.unpack_from("<HBBHBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB", data, model_offset)
# print (model_id,u2,u3,offset_skeleton,u4,u5,u6,u7,u8,u9,u10,u11,u12,u13,u14,u15,u16,u17,u18,u19,u20,num_bones,u22,u23,u24,u25,u26,u27,u28,u29,u30,u31,u32,num_parts,u34,u35,u36,u37,u38,u39,u40,u41,u42,u43,u44,num_animations)
offset_skeleton += model_offset
offset_parts = offset_skeleton + num_bones * 4
offset_animations = offset_parts + num_parts * 32

if model_id == 0x0100:
   object = loadBcx(bcxMap["cloud"])
elif model_id == 0x0302:
   object = loadBcx(bcxMap["ballet"])
elif model_id == 0x0403:
   object = loadBcx(bcxMap["tifa"])
elif model_id == 0x0605:
   object = loadBcx(bcxMap["cid"])
elif model_id == 0x0706:
   object = loadBcx(bcxMap["yufi"])
elif num_parts > 0:
   object = {}
   object["model"] = readModel(data, num_bones, offset_skeleton, num_parts, offset_parts)
   object["animations"] = []
else:
   print "unknown model id: %x" % model_id

object["animations"].extend(readAnimations(data, num_animations, offset_animations))

object_list.append(object)

    return object_list
    
class OpenGLObject:
    def __init__(self, object):
self.object = object
self.time = 0
self.animation = 0
self.draw_normals = False
self.draw_skeleton = True
self.part_map = {}
model = object["model"]
for idx, part in enumerate(model["parts"]):
   self.part_map[part["bone_index"]] = idx

    def nextAnim(self):
self.animation = (self.animation + 1) % len(self.object["animations"])
self.time = 0

    def drawPart(self, part):
glEnable(GL_TEXTURE_2D)

glBegin(GL_TRIANGLES)

for tri in part["tri_color_tex"]:
   for i in xrange(3):
pos = part["vertices"][tri[i][0]]
texcoord = part["texcoords"][tri[i][1]]
col = tri[i][2]

glTexCoord2i(texcoord[0], texcoord[1])
glColor3ub(col[0], col[1], col[2])
glVertex3i(pos[0], pos[1], pos[2])
    
for tri in part["tri_mono_tex"]:
   col = tri[3]
   glColor3ub(col[0], col[1], col[2])
   for i in xrange(3):
pos = part["vertices"][tri[i][0]]
texcoord = part["texcoords"][tri[i][1]]

glTexCoord2i(texcoord[0], texcoord[1])
glColor3ub(col[0], col[1], col[2])
glVertex3i(pos[0], pos[1], pos[2])

glEnd()

glBegin(GL_QUADS)

for quad in part["quad_color_tex"]:
   for i in xrange(4):
pos = part["vertices"][quad[i][0]]
texcoord = part["texcoords"][quad[i][1]]
col = quad[i][2]

glTexCoord2i(texcoord[0], texcoord[1])
glColor3ub(col[0], col[1], col[2])
glVertex3i(pos[0], pos[1], pos[2])

for quad in part["quad_mono_tex"]:
   col = quad[4]
   glColor3ub(col[0], col[1], col[2])
   for i in xrange(4):
pos = part["vertices"][quad[i][0]]
texcoord = part["texcoords"][quad[i][1]]

glTexCoord2i(texcoord[0], texcoord[1])
glVertex3i(pos[0], pos[1], pos[2])

glEnd()

glDisable(GL_TEXTURE_2D)

glBegin(GL_TRIANGLES)

for tri in part["tri_color"]:
   for i in xrange(3):
pos = part["vertices"][tri[i][0]]
col = tri[i][1]

glColor3ub(col[0], col[1], col[2])
glVertex3i(pos[0], pos[1], pos[2])

for tri in part["tri_mono"]:
   col = tri[3]
   glColor3ub(col[0], col[1], col[2])
   for i in xrange(3):
pos = part["vertices"][tri[i][0]]

glVertex3i(pos[0], pos[1], pos[2])

glEnd()

glBegin(GL_QUADS)

for quad in part["quad_color"]:
   for i in xrange(4):
pos = part["vertices"][quad[i][0]]
col = quad[i][1]

glColor3ub(col[0], col[1], col[2])
glVertex3i(pos[0], pos[1], pos[2])

for quad in part["quad_mono"]:
   col = quad[4]
   glColor3ub(col[0], col[1], col[2])
   for i in xrange(4):
pos = part["vertices"][quad[i][0]]

glVertex3i(pos[0], pos[1], pos[2])

glEnd()

# draw normals (I'm not sure about the mono primitives. Maybe they're not lit at all?)
if self.draw_normals:
   global normals
   glBegin(GL_LINES)
   glColor3f(1,1,0)

   for quad in part["quad_mono_tex"]:
for i in xrange(4):
   pos = part["vertices"][quad[i][0]]
   nrm = normals[quad[5]]

   glVertex3i(pos[0], pos[1], pos[2])
   glVertex3i(int(pos[0]+nrm[0]*50), int(pos[1]+nrm[1]*50), int(pos[2]+nrm[2]*50))

   for tri in part["tri_mono_tex"]:
for i in xrange(3):
   pos = part["vertices"][tri[i][0]]
   nrm = normals[tri[4]]

   glVertex3i(pos[0], pos[1], pos[2])
   glVertex3i(int(pos[0]+nrm[0]*50), int(pos[1]+nrm[1]*50), int(pos[2]+nrm[2]*50))

   for quad in part["quad_mono"]:
for i in xrange(4):
   pos = part["vertices"][quad[i][0]]
   nrm = normals[quad[5]]

   glVertex3i(pos[0], pos[1], pos[2])
   glVertex3i(int(pos[0]+nrm[0]*50), int(pos[1]+nrm[1]*50), int(pos[2]+nrm[2]*50))

   for tri in part["tri_mono"]:
for i in xrange(3):
   pos = part["vertices"][tri[i][0]]
   nrm = normals[tri[4]]

   glVertex3i(pos[0], pos[1], pos[2])
   glVertex3i(int(pos[0]+nrm[0]*50), int(pos[1]+nrm[1]*50), int(pos[2]+nrm[2]*50))

   for quad in part["quad_color_tex"]:
for i in xrange(4):
   pos = part["vertices"][quad[i][0]]
   nrm = normals[quad[i][3]]

   glVertex3i(pos[0], pos[1], pos[2])
   glVertex3i(int(pos[0]+nrm[0]*50), int(pos[1]+nrm[1]*50), int(pos[2]+nrm[2]*50))

   for tri in part["tri_color_tex"]:
for i in xrange(3):
   pos = part["vertices"][tri[i][0]]
   nrm = normals[tri[i][3]]

   glVertex3i(pos[0], pos[1], pos[2])
   glVertex3i(int(pos[0]+nrm[0]*50), int(pos[1]+nrm[1]*50), int(pos[2]+nrm[2]*50))

   for quad in part["quad_color"]:
for i in xrange(4):
   pos = part["vertices"][quad[i][0]]
   nrm = normals[quad[i][2]]
   glVertex3i(pos[0], pos[1], pos[2])
   glVertex3i(int(pos[0]+nrm[0]*50), int(pos[1]+nrm[1]*50), int(pos[2]+nrm[2]*50))

   for tri in part["tri_color"]:
for i in xrange(3):
   pos = part["vertices"][tri[i][0]]
   nrm = normals[tri[i][2]]
   glVertex3i(pos[0], pos[1], pos[2])
   glVertex3i(int(pos[0]+nrm[0]*50), int(pos[1]+nrm[1]*50), int(pos[2]+nrm[2]*50))
   
   glEnd()

    def draw(self):
glMatrixMode(GL_MODELVIEW)

glRotatef(180, 1, 0, 0)

model = self.object["model"]
animation = self.object["animations"][self.animation]
skeleton = model["skeleton"]
self.time = (self.time + 1) % len(animation[0])

parent = [-1]
for idx, bone in enumerate(model["skeleton"]):
   ((translation_x, translation_y, translation_z), (rotation_x, rotation_y, rotation_z)) = animation[idx][self.time]

   while parent[-1] != bone[1]:
parent.pop()
glPopMatrix()
   parent.append(idx)
   glPushMatrix()

   if self.draw_skeleton:
glBegin(GL_LINES)
glColor3f(1.0, 0.0, 1.0)
glVertex3i(0, 0, 0)
glVertex3i(0, 0, bone[0])
glColor3f(1.0, 0.0, 0.0)
glVertex3i(0, 0, 0)
glVertex3i(50, 0, 0)
glColor3f(0.0, 1.0, 0.0)
glVertex3i(0, 0, 0)
glVertex3i(0, 50, 0)
glColor3f(0.0, 0.0, 1.0)
glVertex3i(0, 0, 0)
glVertex3i(0, 0, 50)
glEnd()

   glTranslatef(0, 0, bone[0])    
   glRotatef(rotation_y, 0, 1, 0)
   glRotatef(rotation_x, 1, 0, 0)
   glRotatef(rotation_z, 0, 0, 1)
   glTranslatef(translation_x, translation_y, translation_z)

   if bone[2]:
self.drawPart(model["parts"][self.part_map[idx]])

while parent[-1] != -1:
   parent.pop()
   glPopMatrix()

    
def main(*argv):
    pygame.init()
    
    screen = pygame.display.set_mode((640, 480), HWSURFACE|DOUBLEBUF|OPENGL)

    glViewport(0, 0, 640, 480)
    glMatrixMode(GL_PROJECTION)
    glLoadIdentity()
    kFOVy = 0.57735
    kZNear = 1000.0
    kZFar = 3000.0
    aspect = (640.0 / 480.0) * kZNear * kFOVy
    glFrustum(-aspect, aspect, -(kZNear * kFOVy), (kZNear * kFOVy), kZNear, kZFar)

    glEnable(GL_DEPTH_TEST)
    glDepthFunc(GL_LEQUAL)
    glDisable(GL_CULL_FACE)
    glDisable(GL_LIGHTING)
    glDisable(GL_BLEND)
    glPolygonMode(GL_FRONT_AND_BACK, GL_LINE)
        
    glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_REPLACE)

    filePath = os.path.abspath(argv[0])

    # get list of BCX, which should be in the same dir as the .bsx
    # (This is done this way to support operating sytems with a case-sensitive file systems.)
    global bcxMap
    bcxMap = {}
    fieldInfo = {}
    dir = os.path.dirname(filePath)
    for path in os.listdir(dir):
file = os.path.splitext(path)
        if file[1].lower() == ".bcx":
   bcxMap[file[0].lower()] = os.path.join(dir, path)
elif path.lower() == "field.bin":
   fieldInfo["field.bin"] = os.path.join(dir, path)
elif path.lower() == "field.tdb":
   fieldInfo["field.tdb"] = os.path.join(dir, path)

    # get normals
    fieldBin = loadFieldBin(fieldInfo["field.bin"])
    global normals
    normals = []
    for i in xrange(240):
n = struct.unpack_from("<hhh", fieldBin, 0x3f520 + i * 8)
normals.append(vecNormal(n))

    # load BCX or BSX file
    if os.path.splitext(os.path.basename(filePath))[1].lower() == ".bcx":
data = [loadBcx(filePath)]
    else:
data = loadBsx(filePath)

    clock = pygame.time.Clock()
    key_up = key_down = key_left = key_right = False
    rot_x = 0.0
    rot_y = 0.0
    cur_model = 0
    object = OpenGLObject(data[cur_model])
    wireframe = True
    while True:
for event in pygame.event.get():
   if event.type == QUIT:
exit()
   elif event.type == KEYDOWN:
if event.key == pygame.K_UP:
   key_up = True
elif event.key == pygame.K_DOWN:
   key_down = True
elif event.key == pygame.K_LEFT:
   key_left = True
elif event.key == pygame.K_RIGHT:
   key_right = True
elif event.key == pygame.K_ESCAPE:
   exit()
elif event.key == pygame.K_a:
   object.nextAnim()
elif event.key == pygame.K_n:
   object.draw_normals = not object.draw_normals
elif event.key == pygame.K_s:
   object.draw_skeleton = not object.draw_skeleton
elif event.key == pygame.K_m:
   cur_model = (cur_model + 1) % len(data)
   del object
   object = OpenGLObject(data[cur_model])
elif event.key == pygame.K_w:
   wireframe = not wireframe
   if wireframe:
       glPolygonMode(GL_FRONT_AND_BACK, GL_LINE)
   else:
       glPolygonMode(GL_FRONT_AND_BACK, GL_FILL)
   elif event.type == KEYUP:
if event.key == pygame.K_UP:
   key_up = False
elif event.key == pygame.K_DOWN:
   key_down = False
elif event.key == pygame.K_LEFT:
   key_left = False
elif event.key == pygame.K_RIGHT:
   key_right = False
   
if key_up:
   rot_x += 1.0
elif key_down:
   rot_x -= 1.0
   
if key_left:
   rot_y += 1.0
elif key_right:
   rot_y -= 1.0
   
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT)

glMatrixMode(GL_PROJECTION)
glLoadIdentity()
glFrustum(-aspect, aspect, -(kZNear * kFOVy), (kZNear * kFOVy), kZNear, 100000.0)
glTranslatef(0, 0, -2000)
glRotatef(rot_y * 360.0 / 256.0, 0.0, 1.0, 0.0)
glRotatef(rot_x * 360.0 / 256.0, 1.0, 0.0, 0.0)

glMatrixMode(GL_MODELVIEW)
glLoadIdentity()

object.draw()
    
clock.tick(60)
pygame.display.flip()

if __name__ == "__main__":
    main(*sys.argv[1:])
You'll need pygame and the python-opengl bindings. Run with a BCX or BSX file as argument, w toggles wireframe, n toggles normals, s toggles the skeleton, m cycles through models and a cycles through animations.
It should be quite easy to take it from here and make an importer for Blender, but if you're after efficient rendering or animation then Akari's solution in the q-gears source is more efficient.

[Akari]

2Micky: By the way did you look at KAWAI opcode? it is very much related to models and animations.

[Micky]

2Micky: By the way did you look at KAWAI opcode? it is very much related to models and animations.

Not yet. I'll have to add texture support and some basic lighting first. I did check out the stuff from svn, though.

[Akari]

2Micky: By the way did you look at KAWAI opcode? it is very much related to models and animations.

Not yet. I'll have to add texture support and some basic lighting first. I did check out the stuff from svn, though.

I understand texture completly, so look into svn. just need to fing BLINKing routine.

I update SVN daily so stay tuned. Just got to function that calculates lighting.

[Akari]

After a bit of more exploration it looks like the unused byte after each vertex colour could be the normal index. The index has similar values as the vertex index (i.e. if two vertices have the same indices, the corresponding unused values are the same as well), but the normal table doesn't seem to be in the model. Maybe it is shared across all models and stored in the executable?
Another bit: The index goes from 0 to 240 for most BCX. Only Red 13 has a few less.
It's of course possible as well that the value directly encodes the value, and is split into two angle values.

You are right! This is index in normal's table. It's stored at 800df520 address (in executable, right after cos/sin table). It used to calculate final vertex color. And thgere is only 240 values (0x00-0xEF).

[Micky]

You are right! This is index in normal's table. It's stored at 800df520 address (in executable, right after cos/sin table). It used to calculate final vertex color. And thgere is only 240 values (0x00-0xEF).

I've seen it in emulator memory, though it doesn't seem to be in the main executable. I just graphed it in Open Office and it looks very regular. Maybe a subdivision scheme?

[Akari]

You are right! This is index in normal's table. It's stored at 800df520 address (in executable, right after cos/sin table). It used to calculate final vertex color. And thgere is only 240 values (0x00-0xEF).

I've seen it in emulator memory, though it doesn't seem to be in the main executable. I just graphed it in Open Office and it looks very regular. Maybe a subdivision scheme?

I'll trace loading later. I finished with model initialization, now switch to ingame animation and model state update.

[Micky]

I found it in field.bin, at offset 0x3f520. I added it to the discussion about field.bin in the Q-Gears section. (Would it make sense to move that thread here?)

Update: I updated the python code above to read in the normals from field.bin. It assumes that the normal table is at a fixed address, so it may break if the file is different in another build. I'm using NTSC-U/C.
I didn't find a test model that uses mono or mono+tex, so I just assumed in those cases it's the forth component of the colour as well.

[Akari]

Almost finish new field model exporter. Now it uses all the files that contain model data (BSX/BCX/TDB/DAT) ^_^
And correctly assign all textures (including eye and mouth)

[Akari]

I upload lastest exporter to svn. Almost all models exported correctly. And there i config file where you can set what to export.
It also can be used as viewer.