The Harvest Moon Consortium

[OldMansBeard]

[QUOTE=Tiberius_Morguhn;8465]Thanks for the quick update - that did the trick.[/QUOTE]
Good. It was just a slip-up with the repackaging. I knew what I'd done wrong as soon as I saw your post.
[QUOTE]
I just got thru cleaning this tileset : http://nwvault.ign.com/View.php?view=Hakpaks.Detail&id=6417

Took about 4 minutes, and it did export all but 3 tiles. Time to re-hak and check the fixed work out.
[/QUOTE]
The summary should tell you why those three failed. Commonest cause is having a tile with two nodes with the same name.
[QUOTE]
Oh, and there is one small/minute bug - if the tile models are in uppercase or mixed case, the tool converts the filenames to all lower case. There is no problem per se in that, but the NWN hak utility is brain dead so that when you try to add the fixed tiles back to the hak, it keeps the upper/mixed cases mdls and also adds the lower case ones as though they were different! May be good to fix the tool to keep the input and output filename case names the same? Thanks again![/QUOTE]
Just to add to what Bannor has posted -

Yes, we know all about the way nwnhak confuses cases. We fell into that pit many times long before CM3. We decided that the only sane thing to do, was to make all corrected files lower case and if we ever found duplicates with some uppercase characters in their names, we would know which of the two to delete.

What I do, when I've got all the tiles in a set to run though without failures, is simply delete all tile mdls and woks from the hak and then drop the new ones in and check that the file count is correct. Then I know there aren't any wrong-case duplicates.

[Tiberius_Morguhn]

Okay, no problem then. Just trying to help with the devtest.

Oh, and in regards to this:

[QUOTE=Bannor Bloodfist;8466]If your case, copy the original files into a directory, copy the new fixed files into same directory, manually eliminate dupes, and THEN rehak. Start with a clean hak and handle all files when doing this.[/QUOTE]

Waaaaaaaay too much work, I'm lazy.

C:\> rename CT1_*.* ct1_*.*

works wonders :p:

[Bannor Bloodfist]

[quote=Tiberius_Morguhn;8468]Okay, no problem then. Just trying to help with the devtest.

Oh, and in regards to this:

Waaaaaaaay too much work, I'm lazy.

C:\> rename CT1_*.* ct1_*.*

works wonders :p:[/quote]

Yep, do that before you run the program, delete the files from the hak, copy the new files over the old files to cover things that were NOT touched... and away you go.

Just remember, that even though a set has passed through CM3, it will probably STILL have errors. CM3 does wonders, but it can not fix it all.

So, we run our sets through an initial test pass after CM 3 has had a chance to fix things. On that test we look for all normal errors etc, in game, and tag the bad tiles for manual fixing. But, at least we don't have 580 tiles with edges set to -501 and +499 that end up giving gaps in game!

Work load is greatly reduced.

Whatever CM 3 can find that it can't fix, well, if we pay attention to that detailed log report, we can quickly fix errors even before the testing team gets their hands on the set.

[OldMansBeard]

[QUOTE=Sharona Curves;8464]i am going to try this on a particularly buggy tileset. tileset name withheld to protect the innocent and guilty. i am curious to see what this does and if it can actually repair this tileset for me. i have zero experience working with tilesets and any tool that can help would be great. there are sooooooo many good tilesets at the vault and yet most of them are so buggy that they are unusable in their current state.[/QUOTE]

Good. Dive in and try it. It won't fix everything in a tileset - if tiles are visually wrong and badly textured, for example, or if tiles don't fit together, it won't know. It takes people to fix things like that. But it fixes a lot of things that people miss.

[OldMansBeard]

[QUOTE=Tiberius_Morguhn;8465]I just got thru cleaning this tileset : http://nwvault.ign.com/View.php?view=Hakpaks.Detail&id=6417

Took about 4 minutes, and it did export all but 3 tiles. Time to re-hak and check the fixed work out.[/QUOTE]

I DL'ed the ct1 tileset and tried it myself. Your computer is twice as fast as mine !

There was only one failure -
ct1_x04_01.mdl : *** Error - ct1_x04_02 is in the wrong file ***

What were the other two ?

Generally, looking at the detailed log at the kind of things it found to fix, I'd judge that this was a good tileset already. 200 fixes per tile is nothing. There are much worse tilesets out there. I've seen ones coming in at 3000-5000 fixes per tile.

Be interesting to see what Sharona finds with her buggy tileset. :rolleyes:

[Tiberius_Morguhn]

[QUOTE=OldMansBeard;8471]I DL'ed the ct1 tileset and tried it myself. Your computer is twice as fast as mine !

There was only one failure -
ct1_x04_01.mdl : *** Error - ct1_x04_02 is in the wrong file ***

What were the other two ?

Generally, looking at the detailed log at the kind of things it found to fix, I'd judge that this was a good tileset already. 200 fixes per tile is nothing. There are much worse tilesets out there. I've seen ones coming in at 3000-5000 fixes per tile.

Be interesting to see what Sharona finds with her buggy tileset. :rolleyes:[/QUOTE]

Yeah, I splurged last year and bought an AMD Opteron 165 which I proceeded to overclock to 2.2GHz from the stock 1.8GHz. It is dual core with 1MB cache per core. CM3 only ran on one of the cores though.

I left all the models in the directory, so it was trying to work on doors as well - doh!

[QUOTE]ct_udoor_01.mdl
*** Error - ct_udoor_01.mdl does not contain a tile model ***
*** Cannot output model - too buggy ***

ct_udoor_02.mdl
*** Error - ct_udoor_02.mdl does not contain a tile model ***
warning - aabb node 02_dwk_wg_closed does not fit tile
warning - aabb node 02_dwk_wg_closed has mis-oriented face(s)
*** Cannot output model - too buggy ***

ct_udoor_03.mdl
*** Error - ct_udoor_03.mdl does not contain a tile model ***
warning - aabb node 03_dwk_wg_closed does not fit tile
warning - aabb node 03_dwk_wg_closed has mis-oriented face(s)
*** Cannot output model - too buggy ***

ct_udoor_04.mdl
*** Error - ct_udoor_04.mdl does not contain a tile model ***
*** Cannot output model - too buggy ***

ct_udoor_05.mdl
*** Error - ct_udoor_05.mdl does not contain a tile model ***
*** Cannot output model - too buggy ***

Total Fixes = 367857[/QUOTE]

CM3 did expose some problems with this set though - the 3x3 barn and the wall gate now have Z-plane issues for their textures.

[lord rosenkrantz]

I am wondering if it'd be possible to add a new feature to this already great tool.

As you all know, when the CleanModels finds a "too buggy tile" (in the tileset I am using it for, it's often two or more meshes with the same name), it writes it down in the logs, from which we can dig out the tiles names to import them in 3dsmax and manually fix them before running again the utility.

I am not sure how much work it'd take, but it'd be great if CleanModels could write a "dummy.set" file, with all internal parameters (corner tile terrain entries for example) set again to a "dummy" value, that would include only the tiles it couldn't export because of their internal errors.

That way, instead of manually picking and importing the 25 tiles from a folder that contains 200 other tiles, we could use Tileset Creator from inside 3dsmax and one-click import them all from the dummy.set file, saving a ton of time in such preliminary task.

The dummy set file wouldn't be any good to paint the tiles in game of course, but would do its dirty work in the tile debugging stage. Obviously, once the tiles have been debugged and exported in a format that CleanModel can work on, we can simply delete the dummy set file and switch back to our proper set file for further work.

How complicate would be to add some code to make the tool output such a simple set file? Would it be worth it?

[OldMansBeard]

Well, I can see where you are coming from, but I don't think it would be very simple to implement inside CM3.

CM3 remembers which tiles are too buggy so it could do something more helpful with the list but it doesn't know anything about .set files so asking it to construct one is non-trival.

Personally, when I'm fixing up the baddies, I do it in notepad so I can see what's really wrong. But that's not to everyone's taste, so let's see what we can do.

Let's approach the problem incrementally. I could make a tiny change in CM3 so that, at the end of the summary, it prints a straight list of the files that were too buggy. (Saves you skimming through the file finding "**"). Or I could write it to another file. Something.

Then we could have an entirely separate program that picks up that list, reads the original .set and outputs a modified one to your liking. Deletes the good tiles and renumbers the rest, or whatever (you'd have to tell me). It would be nice to annote it somehow with what is wrong, so you saw it inside Max alongside the tile somehow but I don't know if that's possible.

So, if we imagine a program that reads a list of buggy tiles, reads in the original .set and outputs your modified .set, would you be able to tell me exactly what changes to the .set you would want?

[lord rosenkrantz]

alright, just quickly before I need to go out again (should have more time this late evening), your post stimulated my brain to a slightly different (and hopefully more efficient) approach.

Although CleanModel3 doesn't understand the .set format, I assume that it is already able to process simple text formats (it handles ASCII format after all), and if so, how about we make it create a copy of the .set file, again renamed for example to "dummy.set", and then make the program process this dummy copy with simple operations. CleanModel could scan the set file, and every time it finds a model name that matches one of those included in the "too buggy" model list that resides in memory, it could replace the line "TopLeft=[any terrain]" with a "TopLeft=Buggy" entry. In this simple form, it would already serve the purpose I was after, as SetCreator can then allow me to load the buggy tiles selecting this dummy terrain called "Buggy". The program should be able to also add a new terrain entry in the terrain list just below the set file header, obviously.
And of course, the option to have another stand-alone little program that is able to process the set format and make changes according to the log output by CleanModel works as great. Which solution is better you are able to tell more than me.

But since the logic would be the same, we could do something a little more refined. We could make a list of the main bug categories that we could encounter, up to eight of them. In the set file in fact, we have 8 free slots that could be used with such approach, as each tilemodel is listed with 4 corner entries and 4 edge entries for terrains and crossers

Such utility should then:
1) Add 4 new terrains and 4 new crossers to the .set file it processes, updating the total count. The first terrain should be "_cant_output". The second could be "_bad_anim", the third "_duplicate", the fourth "_aabb_issue", and the crossers could be "_off_tile", "_cant_load" and other categories that now I can't recall (the tileset I used to run a test didn't have other types of issues)

2) for each tile, replace as appropriate the corner or edge entry with our dummy terrain/crosser in case CleanModel3 has found the correspondant issue while fixing the tile

With such a modified .set file, the actual manual fixing would be a lot easier. Personally I would load all aabb problematic tiles at once for example, as once my eyes are tuned to find walkmeshes issues, I prefer to handle them in sequence. But obviously the first issues to take care of would be the "_can't_load" and "_cant_output" terrain/crosser entries, and only in a second stage I'd take care of the other issues

Again, I don't know how much work it'd take to code such functionalities, I am just assuming the code would follow a logic like if tile X cannot be output, then replace the following "TopLeft=[any terrain]" line with a "TopLeft=_cant_output" line and so on.
We can use a static approach, meaning that instead to proceed to find the first corner/edge "free" entry, we can associate each of the 8 slots to a specific dummy entry, so the program only needs to run a simple text string research to find the line to replace
The result should look like the following (the highlights in the .set would be given by the underscore I use at the beginning of the dummy names)

[FONT="Courier New"][TILE2]
Model=tcn01_a08_01
WalkMesh=msb01
TopLeft=_cant_output
TopLeftHeight=0
TopRight=_bad_anim
TopRightHeight=0
BottomLeft=_duplicate
BottomLeftHeight=0
BottomRight=_aabb_issue
BottomRightHeight=0
Top=
Right=
Bottom=
Left=
MainLight1=1
MainLight2=1
SourceLight1=1
SourceLight2=1
AnimLoop1=1
AnimLoop2=1
AnimLoop3=1
Doors=0
Sounds=0
PathNode=T
Orientation=0
VisibilityNode=J
VisibilityOrientation=90
ImageMap2D=MICN01_A08

[TILE3]
Model=tcn01_a09_01
WalkMesh=msb01
TopLeft=Building
TopLeftHeight=0
TopRight=_bad_anim
TopRightHeight=0
BottomLeft=Water
BottomLeftHeight=0
BottomRight=_aabb_issue
BottomRightHeight=0
Top=
Right=_cant_load
Bottom=
Left=
MainLight1=1
MainLight2=1
SourceLight1=1
SourceLight2=1
AnimLoop1=1
AnimLoop2=1
AnimLoop3=1
Doors=0
Sounds=0
PathNode=T
Orientation=0
VisibilityNode=A
VisibilityOrientation=0
ImageMap2D=MICN01_A09[/FONT]

In fact, a tile with a "_cant_load" entry shouldn't have any other dummy terrain/crossers, but it's just for sake of example


-----------------


Anyway, I think such approach would address two things you pointed out:

[QUOTE]Deletes the good tiles and renumbers the rest, or whatever (you'd have to tell me). It would be nice to annote it somehow with what is wrong, so you saw it inside Max alongside the tile somehow but I don't know if that's possible.[/QUOTE]

We wouldn't need to delete and renumber any tiles, saving the trouble to also renumber the group references in the dummy.set
And the dummy terrain/crossers that are assigned to each tile would tell us at least what category of issues we need to look for, and they'd do it immediatelly and from inside 3dsmax itself

[OldMansBeard]

Okay, but I would go for a separate program because once you can read and write set files, you might as well check them for correctness and that's a whole new ball-game not related to the tile models themselves.

These are the error conditions that make CM3 unable to export a model. You won't have encountered them all and you might never encounter some of them but they have all turned up somewhere in the tilesets we've worked on.

load failed
The file is garbled or contains (partial) nonsense. It probably won't even import meaningfuly into Max but it may be possible to rescue it manually in notepad. Or it might be a compiled file by mistake.

Bad errors

• The file doesn't contains a model at all
• The file contains a model but it isn't a tile
• The file contains a tile but it's the wrong one
• The file contains the same tile model twice
• The model contains duplicate nodes
• A node has no parent declared
• A node has a parent declared but that parent node does not exist
• The parent node exists but is duplicated and therefore ambiguous
• Duplicated animations
• Danglymesh node with the wrong number of constraints
• Animmesh node with the wrong number of animverts
• Animmesh node with the wrong number of animtverts
• The aabb node has duplicate faces
• Walkmesh geometry is so bad that aabb data cannot be built

So there are too many to represent by the four corners but we could group them, perhaps, and that would help.

We ought to group them according to what kind of operation you would do to fix it but you would have to tell me that. I know how I would fix them but that's not necessarily a good guide.